Memory fix in your crawler?

[matthewmueller]

I saw this comment from you on Stack Overflow to this question:

Thanks to your suggestion I think I got to the bottom of the issue. It's quite strange and I'm not sure is that a bug or feature. I store all URLs the crawler visited in an object. This is a tiny memory leak on its own but insignificant. The strange thing is that the string is somehow related to the whole HTML object and keeps reference to it. I have to add a character to it and than remove it to destroy the reference. The typeof(string) is string not object so I'm surprised that any references are maintained. Anyway... thank you for help. –

I was wondering if you can provide information on how you tested for this or discovered this? Is there a spot in the code of node-web-crawler where you incorporated this fix?

[lukaszkujawa]

I can vaguely remember that I dump memory after 1, 5 and 10 minutes of running the software and later analysed it with Google Chrome. I think the problem was with the "jQuery" library I've been using at that time. I replaced it with a different library and it was OK. Sorry but it was long time ago and I don't remember details.