(Use a version of the ac factoring method.)
#3x^2 - xy - 24y^2#
The product of the first and last coefficients is
#3xx-24 = -72#
We want to split the middle term #-xy# into two terms. The coefficient of this middle term is #-1#
So, we need two numbers whose product is #-72# and whose sum is #-1#.
#1xx-72# no, they do not add to #-1#
#2xx-36# no
#3xx-24# no
#4xx-18# no
#5xx# no
#6xx-12# no
#7xx# no
#8xx-9# yes! the product is #-72# and the sum is #-1#
Now split the middle term using #8xy# and #-9xy# (In the end it won't matter which order you write them in, so let's just use the one above.)
#3x^2 - xy - 24y^2#
#3x^2 +8xy - 9xy - 24y^2# Now factor by grouping.
#(3x^2 +8xy)+( - 9xy - 24y^2)#
# = x(3x+8y)+(-3y)(3x+8y)#
# = (x-3y)(3x+8y)#
Check by multiplying:
#3x^2+8xy-9xy-24y^2# looks good, so we are finished.
