Vacations!

And hence start the summer vacations!

I'm really happy that they have finally arrived. Was getting tired of this last semester and some time out will surely be well used to refill batteries before the final semester begins.

Just wish it had gone better. At least I'm proud to say that I've learned quite a lot of new things throughout these 12 weeks. Some were good, some were bad and all of them were important to my life in one way or another.

The most memorable detail is the fact that I no longer stay away from Java. I've always avoided it, don't like eclipse, it's slow as heck compared to native binaries building a user interface is a real shame when compared to compilers for Windows that have been available for almost a decade now.

However, it does bring some fancy new advantages available nowhere else. We can indeed code once and run everywhere if things are done right. The code that runs on a desktop can also run from a server on the command line.

And.. using Netbeans as an IDE was indeed a nice touch of fresh air to escape a bulky, buggy and slow code editor.

Another nice memory from this semester is the independent study. Not only I was able to do a project from a professional perspective as it was also fun and something that I'll continue working through the next months.

For the vacations, many other projects await. Time will pass really quickly and I also want to rest. From here up to December things will not get simpler.

Luckily, from a financial perspective I'm finally at ease. Not only have I managed to secure my full payment of the MSE degree and survive the final months, as I'm also increasing my current savings and doing investments for the future.

Things haven't been easy but they're moving forward.

Avoiding stackOverflow errors in Java

I have coded a method in Java that will recursively iterate through all folders and respective sub-folders on disk.

The code seemed to work as expected, but whenever reaching a certain folder it would simply thrown an exception error and complain about stackOverflow.

Now, what is this stack overflow all about? Googling around it seems to occur whenever you enter into an endless loop situation.

This occurred while crawling sub-folders, so even thought my hard drive is quite filled up, it's not exactly filed with folders up to infinity.

So, what is wrong on this picture?

It turns out that this method is vulnerable to badly formed dynamic links. Meaning that whenever a link is found that points to a folder on lower level - it would just re-bounce back to that lower sub-folder and then loop back again ad eternum.

I've lost plenty of time trying to avoid dynamic links from being crawled but to no avail. Also reached the point of calling the absolute path of dynamic links and indexing each path on a database to check one by one if they had already been called to avoid these annoying loops (losing a lot of performance in the process).

Fortunately, found the solution to this riddle on this website - http://leepoint.net/notes-java/io/10file/20recursivelist.html

Albeit being a simplistic code, it provides a very efficient way to deal with the "symbolic-link-limbo". All you need to is define a depth level. They mention 20 as the default value and I applied the same concept on my code.

The second piece of the puzzle is verifying that each new directory that you want to crawl matches in terms of absolute path and canonical path. I've used the code from the following page as inspiration: http://www.idiom.com/~zilla/Xfiles/javasymlinks.html

After applying these changes, the code WORKED LIKE A CHARM!

Now the same method is indexing all folders up to 20 levels of depth and working as expected without seeing any more stack overflow messages. Under a MacBookPro it can index over 600 000 files under 16 minutes using less than 10Mb of heap space in RAM.

:)