Note: There are other ways of doing this (of thinking about it). This is one way.
Remember that #(ax+b)(cx+d)=acx^2+(ad+bc)x+bd#
So the first two coefficients of the factors have to multiply to give us the number in front #x^2#.
Example: for factoring #6x^2+#some other stuff,
the possibilities are #(6x+b)(x+d)# or #(2x+b)(3x+d)#.
(We don't need to worry about #(3x+b)(2x+d)#. because it's the same as the second choice with #b# and #d# exchanged.)
The constants have to multiply to give us the constant in the original expression.
Example: For #6x^2+11x-10#, the choices for how to end are #(ax+10)(cx-1)# or the other way around #(ax-10)(cx+1)#
(There's a difference if #a!=b# because the 'minus' sign could go with either the #10# or the #1#.)
OR #(ax+5)(cx-2)# or #(ax-5)(bx+2)#
(Again the 'minus' could go with the #5# or with the #2#.)
Finally, we also need #ad+bc =# the coefficient of #x#,
which in this example is #11#. That's where the "trial" happens. We need to check each of the possibilities.
Here they are:
#(6x+10)(x-1)# or #(6x-10)(x+1)#
#(2x+10)(3x-1)# or #(2x-10)(3x+1)#
#(6x+5)(x-2)# or #(6x-5)(x+2)#
#(2x+5)(3x-2)# or #(2x-5)(3x+2)#
The faster you can do ad+bc (O+I, if you think FOIL),
the faster you can do the checking.
First line, I get #6# and #10#, I'm not going to get #11# from those.
Second line #2# and #30#, no! I won't get #11# from them.
Third line: #12# and #5#, no good.
Fourth line #4# and #15# -- possible!. The difference is 11, so check further: #(2x+5)(3x-2)# gives us #-4# and #+15#, and those add up to #11#. That's the one we want.
To be certain, multiply again:
#(2x+5)(3x-2)=6x^2-4x+15x-10=6x^2+11x-10#.
So that works.
Remember you can't always factor by trial and error. #x^2+2x+7# won't factor using whole numbers.
The only possibilities would be #(x+1)(x+7)# of #(x-1)(x-7)#. But neither of those work.
So #x^2+2x+7# is "prime" or irreducible" or "not factorable" -- depending on what you've been taught to call it.
(If you learn enough math you may learn more about this.)