1
Okay, I just figured this out. I believe this is what you do: we know solution is spherically symmetric by the hint and because the boundary conditions show that the solution has no $\theta$ or $\phi$ dependence. So converting to spherical coordinates, the problem looks like:
$$u_{tt}-u_{rr}-\frac{2}{r} u_r =0$$
We separate variables: $u(x,t)= T(t) X(x)$. So
$$\frac{T_{tt}}{T}-\frac{R_{rr}}{R}-\frac{2}{r}\frac{R_r}{R}=0 \implies \\
\frac{T_{tt}}{T}=-\lambda \\
\frac{R_{rr}}{R}+\frac{2}{r}\frac{R_r}{R}+\lambda = 0$$where $\lambda \in R$
We know $$T(t)=A\sin(\sqrt\lambda t) +B\cos(\sqrt\lambda t)$$ The initial condition $$u_t(r,0) = 0 \implies T_t(t) = 0 \implies T(t)=B\cos(\sqrt\lambda t)$$
We also have $$\frac{R_{rr}}{R}+\frac{2}{r}\frac{R_r}{R}+\lambda = 0$$ By the hint, we write this as $$(rR)'' = -\lambda (rR)$$ Making a change of variables with $$L(r) =rR(r)$$ we get
$$L_{rr} = -\lambda L \implies L(r)=E\sin(\sqrt\lambda r)+F\cos(\sqrt \lambda r) \implies R(r) = \frac{E\sin(\sqrt\lambda r)}{r}+\frac{F\cos(\sqrt \lambda r)}{r}$$ We rule out the cos term because we get a 1/0 as r approached infinity, which is bad. So
$$R(r) = \frac{E\sin(\sqrt\lambda r)}{r}$$
Now we combine the R(r) and T(t) solutions. We call $n = \lambda$ for the sake of notation.
$$u(r,t)=\sum_{n= 1} ^{\infty} \frac{C_n \cos(nt) \sin(nr)}{r}$$
By the initial condition (equation 2) in the problem, we have $$\begin{align} &u(r,0)=f(r)=\left\{\begin{aligned} &1\quad &&r<1,\\ &0 &&r\ge 1,\end{aligned}\right.\qquad \end{align}$$
So we have
$$u(r,0)=\sum_{n= 1} ^{\infty} C_n \sin(nr) = r f(r)$$ This is a sine fourier series with $L=\pi$, so
$$C_n = \frac{2}{\pi} \int_0^{\pi} sin(nr) r f(r) dr = \frac{2}{\pi} \int_0^1 sin(nr) r dr $$
We use integration by parts here and get
$$C_n = \frac{2}{\pi} \frac{\sin(n)-n\cos(n)}{n^2}$$
So
$$u(r,t)=\sum_{n= 1} ^{\infty} \frac{\frac{2}{\pi} \frac{\sin(n)-n\cos(n)}{n^2} \cos(nt) \sin(nr)}{r}$$
$$u_{tt}-u_{rr}-\frac{2}{r} u_r =0$$
We separate variables: $u(x,t)= T(t) X(x)$. So
$$\frac{T_{tt}}{T}-\frac{R_{rr}}{R}-\frac{2}{r}\frac{R_r}{R}=0 \implies \\
\frac{T_{tt}}{T}=-\lambda \\
\frac{R_{rr}}{R}+\frac{2}{r}\frac{R_r}{R}+\lambda = 0$$where $\lambda \in R$
We know $$T(t)=A\sin(\sqrt\lambda t) +B\cos(\sqrt\lambda t)$$ The initial condition $$u_t(r,0) = 0 \implies T_t(t) = 0 \implies T(t)=B\cos(\sqrt\lambda t)$$
We also have $$\frac{R_{rr}}{R}+\frac{2}{r}\frac{R_r}{R}+\lambda = 0$$ By the hint, we write this as $$(rR)'' = -\lambda (rR)$$ Making a change of variables with $$L(r) =rR(r)$$ we get
$$L_{rr} = -\lambda L \implies L(r)=E\sin(\sqrt\lambda r)+F\cos(\sqrt \lambda r) \implies R(r) = \frac{E\sin(\sqrt\lambda r)}{r}+\frac{F\cos(\sqrt \lambda r)}{r}$$ We rule out the cos term because we get a 1/0 as r approached infinity, which is bad. So
$$R(r) = \frac{E\sin(\sqrt\lambda r)}{r}$$
Now we combine the R(r) and T(t) solutions. We call $n = \lambda$ for the sake of notation.
$$u(r,t)=\sum_{n= 1} ^{\infty} \frac{C_n \cos(nt) \sin(nr)}{r}$$
By the initial condition (equation 2) in the problem, we have $$\begin{align} &u(r,0)=f(r)=\left\{\begin{aligned} &1\quad &&r<1,\\ &0 &&r\ge 1,\end{aligned}\right.\qquad \end{align}$$
So we have
$$u(r,0)=\sum_{n= 1} ^{\infty} C_n \sin(nr) = r f(r)$$ This is a sine fourier series with $L=\pi$, so
$$C_n = \frac{2}{\pi} \int_0^{\pi} sin(nr) r f(r) dr = \frac{2}{\pi} \int_0^1 sin(nr) r dr $$
We use integration by parts here and get
$$C_n = \frac{2}{\pi} \frac{\sin(n)-n\cos(n)}{n^2}$$
So
$$u(r,t)=\sum_{n= 1} ^{\infty} \frac{\frac{2}{\pi} \frac{\sin(n)-n\cos(n)}{n^2} \cos(nt) \sin(nr)}{r}$$