Consider the top of the cone touching the inner sphere and consider the height of the cone h
then [h−R],[ R=radius of sphere] will equal one leg of a right angle triangle, the other two sides of the triangle would be r the radius of the cone that touches the sphere and the radius of the sphere, R.
So we can say,[h−R2]+r2=R2, thus r2=R2−[h−R2]
Expanding the bracket and rearranging, r2=[2Rh−h2]......[1]
The volume of a cone isπ3[r2h], substituting for r in ....[1]
#V=pi/3[2Rh-h^2]h....=pi/3[2Rh^2-h^3]# ........#[2]#
Differentiating [2] with respect to h, keeping R constant,
DV/dh=4πRh3-3πh23=0 for max/min.........[3]
evaluating [3] yields h=43R, and this value of h will maximise the volume of the cone. Taking the second derivative and substituting for h=43r will give a negative result confirming this value of h will ensure the maximum volume of the cone.
Substituting for h=43R, in ...[2] volume of cone will equal,
π3[2πR[4R3]2−[4r3]3]. Evaluated this will give the above answer. I hope this was helpful , and I will request someone to check the answer.