I found the issue occurred when after recording I used the wizard which wanted to validate my server credentials. I entered "NTLM" under realm and then my username and password. When I used the test connection feature in the wizard, it validated it in the wizard but then subsequently gave me all the 401 errors in the check user path feature.
When I changed it to remove the "NTLM" under the realm field from the server (which the wizard was showing as authentication failed when I tried the test connection) then when I did the check user path playback everything worked as expected.
Is this a bug in 5.2.4? What are you meant to enter in the realm field in the wizard.