PDA

View Full Version : Getting Correct URL With Server Load Distributions?



NeoMorph
October 27th, 2007, 02:31 AM
OK I've been scraping from allmusic.com and have come across a problem.

When you go to www.allmusic.com (http://www.allmusic.com) it opens up the page and the url changes to something like wm10.allmusic.com or wm06.allmusic.com. Anyone have any ideas how to grab this prefix change in Girder.

I think I can do it in NR as I have access to the url variable but I'm doing the processing in Girder so I need to do it there. Is there access to the returned url available in Girder too?

NeoMorph
October 27th, 2007, 03:19 AM
I finally got around it by adding a hidden web browser in NR and Girder can set the URL and then a RegisterVariableWatch looks at the url and when it changes to the new URL it sends it back to Girder.

A bit convoluted but it works...