Problem 5

Let $m=f(1)$; by step 5, $m>0$. If $m\le1$, apply the full form of $(*)$ with $x=\sqrt{1-m},y=1$: $f(1-m+f(1))=f(1-m)+f(1)+2f(\sqrt{1-m})$ becomes $f(1)=f(1-m)+f(1)+2f(\sqrt{1-m})$, so $f(1-m)=-2f(\sqrt{1-m})\le0$; combined with $f(1-m)\ge0$, $f(1-m)=0$, so $1-m=0$, i.e. $m=1$. If $m\ge1$, then by step 1, $f(m)=f(f(1))=f(1)=m$, so $f(m^2)=f(f(m))=f(m)=m$; apply the full form of $(*)$ with $x=\sqrt{m^2-m},y=1$: $f(m^2-m+f(1))=f(m^2-m)+f(1)+2f(\sqrt{m^2-m})$ becomes $f(m^2)=f(m^2-m)+m+2f(\sqrt{m^2-m})$, i.e. $m=f(m^2-m)+m+2f(\sqrt{m^2-m})$, so $f(m^2-m)=-2f(\sqrt{m^2-m})\le0$; combined with $f(m^2-m)\ge0$, $f(m^2-m)=0$, so $m^2-m=0$, and since $m\ge1$, $m=1$. Either way, $f(1)=1$.