Your second equation #atimesb=9# can be rewritten as #a=9/b#. Put this into the first equation:
#7times(9/b)-5timesb=6#
#63/b - 5b =6#
Multiply all terms by b.
#63 - 5timesb^2 - 6timesb = 0#
#-5b^2 -6b + 63=0# solve this equation
#Delta=36 - 4times(-5)times(63)#
#Delta = 1296#
#(Delta)^0.5 = 36#
Solution for b is #=[-(-6)-36] /(2times-5)#
#b=30/(-10) = -3#
Therefore if b is -3 a is also -3 (since #a=9/b = 9/-3 = -3#). This is the first solution (a=b=-3).
The value of #343times(a^3) - 125times(b^3) = 343times(-3^3) - 125times(-3^3) = -5886#
There is another solution
#b=[-(-6) + 36] /(2times-5)#
#b=42/-10 = -4.2#
#atimes-4.2=9#
#a=-2.1428#
This is the second solution. (a=-2.1428 and b=-4.2).
In this case, your answer is #343times(-2.1428^3) - 125times(-4.2^3) = 5886#