Inaccuracy on LZX documentation

A few more days of debugging passed.

I really appreciate to have documentation available from Microsoft so that I can give somewhat correct names to the binary structures.

But progress moves slowly as the available pages only provide twitter-style sentences to explain how each block tie together and even these are sometimes misleading. Nevertheless, I can't complain much as progress is slow but not frozen.

Right now I've reached to a point where an implementation of the Microsoft LZX-2 algorithm is required to compress data. Looking at alternative implementations made by others such as the WINE team, I found this funny comment:

/* LZX decruncher */

/* Microsoft's LZX document and their implementation of the

* com.ms.util.cab Java package do not concur.

*

* In the LZX document, there is a table showing the correlation between

* window size and the number of position slots. It states that the 1MB

* window = 40 slots and the 2MB window = 42 slots. In the implementation,

* 1MB = 42 slots, 2MB = 50 slots. The actual calculation is 'find the

* first slot whose position base is equal to or more than the required

* window size'. This would explain why other tables in the document refer

* to 50 slots rather than 42.

*

* The constant NUM_PRIMARY_LENGTHS used in the decompression pseudocode

* is not defined in the specification.

*

* The LZX document does not state the uncompressed block has an

* uncompressed length field. Where does this length field come from, so

* we can know how large the block is? The implementation has it as the 24

* bits following after the 3 blocktype bits, before the alignment

* padding.

*

* The LZX document states that aligned offset blocks have their aligned

* offset huffman tree AFTER the main and length trees. The implementation

* suggests that the aligned offset tree is BEFORE the main and length

* trees.

*

* The LZX document decoding algorithm states that, in an aligned offset

* block, if an extra_bits value is 1, 2 or 3, then that number of bits

* should be read and the result added to the match offset. This is

* correct for 1 and 2, but not 3, where just a huffman symbol (using the

* aligned tree) should be read.

*

* Regarding the E8 preprocessing, the LZX document states 'No translation

* may be performed on the last 6 bytes of the input block'. This is

* correct. However, the pseudocode provided checks for the *E8 leader*

* up to the last 6 bytes. If the leader appears between -10 and -7 bytes

* from the end, this would cause the next four bytes to be modified, at

* least one of which would be in the last 6 bytes, which is not allowed

* according to the spec.

*

* The specification states that the huffman trees must always contain at

* least one element. However, many CAB files contain blocks where the

* length tree is completely empty (because there are no matches), and

* this is expected to succeed.

*/

http://source.winehq.org/source/dlls/itss/lzx.c

Funny because a decade has passed and still we see specifications for other formats to contain lapses, mistakes and misplacements on official docs from the MS corporation. Even the patent claim they made for the WIM specification contains inaccuracies.

Would be nice to see things change.

Inaccuracy on WIM documentation

For those brave souls in the future, trying to interpret the WIM headers using the documentation provided by Microsoft: there is a typo on the declaration of data structure for RESHDR_BASE_DISK.

Where one reads:

typedef struct _RESHDR_BASE_DISK

{

ULONGLONG ullSize;

BYTE sizebytes[7];

LARGE_INTEGER liOffset;

}

It should actually be read as:

typedef struct _RESHDR_BASE_DISK

{

BYTE bFlags;

BYTE sizebytes[7];

LARGE_INTEGER liOffset;

}

The only difference is replacing ullSize by bFlags. If you don't, the difference is that a BYTE on this case is only sized in 8 bits whereas ULONGLONG is sized in 64 bits.

If you're trying to read the header from a binary file then you'd be stuck with the wrong results.

I had actually noted this detail over a year ago. Now I was looking at this again and had to spent around two days doing the math and printing the hex dump to see why things were not looking right (and get some grey hairs).

So, now I've decided to write it once for all in the blog so that it won't get forgotten again. If it helped you, do let me know.

Happy Christmas!

:)

Fresh start on the Alexa rank

I like watching the Alexa rank to get a notion of how different sites fare in terms of mainstream audience.

Wonko, "the Sane" has a very different opinion about the usefulness of this rank. Yet, it is indeed interesting to observe how we are starting from a fresh new domain that was ranked in 2 million about a week ago and right now is already breaking the top 30 000 barrier.

Not something that we get to see very often.

More interesting to note that over the past weeks our rank had been around 60 000 and that moving to a new domain has also impacted our relevance to search engines as an old domain has far more trust when compared to a brand new one, bringing less visitors from google and the sort.

I'm just glad that this transition is going so smoothly. A lot of things could have been gone wrong and still a lot more in need to be fixed but so far, I'm really happy to see how we are going.

:)

Support for Android and Iphone

One of the advantages from upgrading the board is that we also enjoy some of the recent changes.

Here is one of the nice surprises, support for mobile browsing already available by default:

The Reboot empire.

Boot Land was rebooted, why?

Over the past few months, anyone could note that despite our growth in terms of popularity, page views and visits - we were no longer working true to our founding principles.

Our once peaceful netizens turned the public forums onto a circus stage/arena for proceeding in never-ending quarrels.

While looking back, I see how much energy was wasted in defending or destroying opposing positions between aristocracy members without practical results. Instead of seeing progress, I'd risk stating that we actually saw regress and crisis to install in our development/research projects across 2010.

-------------------------

The Boot Land republic

The flagship weapon of our community, Winbuilder, saw script warlords requesting so many new features to a script engine craftman that seldom times said no. Encouraged with a multitude of feedback and requests, he began an effort to instantiate syntax correctness that would last the following two years.

From my perspective, this craftman was (and is) well intended. But each new version would disrupt scripts coded in older versions of the weapon. This forced warlords to re-train themselves and update all scripts in weaponry stock.

The script warlords were (and are) well intended. They know that winbuilder is one of the strongest weapon on their arsenal. They desire new specialized features that may give them an advantage on battlefields not just in open plains as before, but also in mountains, swamps and tropical environments.

I would have preferred to see other weapons being used together with winbuilder to achieve optimal performance in combat, rather than seeing both the engine craftman and these warlords creating a tool that served their specific situation alone. Since design simplicity was no longer present, we really lost the single most important combat advantage.

The engine kept on convolving to a reality each time farther apart from practical reality in present battle fields. While warlords of conquered domains kept on using older versions and consider other weapons for conquering new territories, new versions became ignored or deeply criticized.

Personal conflicts escalated to unprecedented levels of animosity to find guilt in others.

Rage settled in, projects are removed from public sight, opposing parties verbally attack each other on sight of public movement. This discourages thousands of netizens from joining the public forum. Vengeance, rather than reason, becomes a frequent dish served at public gatherings that are now only frequented by a few surviving senators that observe, but seldom times intervene in fear of retaliation.

Ironically enough, due to the plural investment in many other projects instead of just being a winbuilder centric community, we also saw the Boot Land domain escalate to an unparalleled growth across the boot disk universe at the Internet.

We see for the first time the barrier of 700 000 page views being broken and also celebrate the success of many excellent projects promoted by unaligned brave souls at our community, that put their heart in the work and move the boot disk state of the art by themselves.

Conflicts between aristocratic members stale any decision or course of action for the future.

Our online republic fails.

-------------------------

The reboot empire begins

Situations of this kind are not uncommon to occur at any community of reasonable dimension. I remember clearly a sequence of similar events that took place at 911CD.net some years ago. Bart, the author of BartPE superseded by far in popularity the work of DoctorXP, author of the 911CD project.

DoctorXP stepped down from public activity and BartPE became the defacto tool in coming years. Bart lost interest after some years and his work was left to other initiatives such as Reatogo and ubcd4win. Conflicts soon started to fringe the once peaceful environment at 911CD.net to a stage of pandemonium until everyone was unhappy.

As time passed, our state of conflict starts to resemble each time closer to the one observed at 911CD.

Since nobody was accepted as right by others nor admitted wrong doing on their side, decisions still need to be made and the state of Imperium was declared.

The reboot empire comes to life.

Martial law is instantiated to restore a sense of order amidst the political chaos. Those who cross the line of civilized manners are handled summarily regardless of their rank in society. A sad period but necessary to prevent our public forum from returning onto a public arena.

We live in the age of pax romana.

The goal is clear. We work to rebuild stability, to define the new milestones of expansion for our domain and to ensure that our society regains once again it's own balance to conquer new territories. These decisions will surely not please everyone but we are a breed of fighters.

We reboot.

RawReg included in pwning bootkit

Just of hearing the name of RawReg brings back some really good memories from the attic.

So, it was kind of fun to read IceCube mentioning that it was included by default on the Stoned bootkit, a project described by the author as:

Stoned Bootkit is a new Windows bootkit which attacks all Windows versions from 2000 up to 7. It is loaded before Windows starts and is memory resident up to the Windows kernel. Thus Stoned gains access to the entire system. It has exciting features like integrated file system drivers, automatic Windows pwning, plugins, boot applications and much much more. The project is partly published as open source under the European Union Public License. Like in 1987, "Your PC is now Stoned! ..again".

Peter Kleissner, Software Developer in Vienna

The project can be found at http://stoned-vienna.com/

-------------

Well, this certainly brought me back good memories when I didn't worried about the integration of enterprise applications and their survivability in the long term.

Here is a screenshot of the "about" screen on rawreg, while running under Wine on the Mac OS.

Looking forth to the future, many plans lay ahead. However, I still question every day if there will ever be time and commitment to see them through.

700 000 page views

For the first time since its inception in 2006, Boot Land has reached a new record with more than 700 000 page views served on a single month.

This means that over the course of 11 months, we have successfully grown more than 77% in terms of popularity when compared to last year.

Over the same period of time, the daily consumption of bandwidth on the server has surpassed 100Gb per day while also reducing our RAM usage to little above 4Gb.

Things are looking bright when looking at these numbers and indeed there are many good reasons to be proud about, however, not all news are roses as seen on the case of the winbuilder wars.

The good part is that our community has finally outgrown this state of warfare and other projects are also giving very solid signs of growth as well. Across 2010 we can see grub4dos, Sardu, multiPE, Wimb's work and many other good projects rising to become top tools in this industry. These projects are indeed becoming the defacto tools on this arena.

At this rate, 2011 is already promising to become a really interesting year in terms of community achievements.

:)

To EJB, or not to EJB?

I've found myself asking this question, what advantages does EJB bring?

Googled a lot, found a lot some bulleted lists repeated across many sites, but not so easy to find the real reasons that might drive a person to consider EJB in regard to other options.

Finally, I've found a really a good article from Humphrey Sheil that was written at the year of 2000. It encompasses the fundamental questions that one should ask ourselves while looking at this technology even after a decade has passed.

Below is the introduction:

To EJB, or not to EJB: that is the question.

Whether 'tis nobler in the mind, to suffer

The slings and arrows of outrageous licensing;

Or to take arms against a sea of potential overheads and features,

And by opposing end them? To roll your own: to reinvent the wheel;

No more; and by reinvent, to say, we continue

The heart-ache of low-level systems maintained in-house,

and the thousand natural shocks

That flesh is heir to; 'tis a consummation

Devoutly to be avoided.

You can read the whole article at this location: http://www.javaworld.com/javaworld/jw-12-2001/jw-1207-yesnoejb.html?page=1

Disabling SVN on Windows

Windows has some fascinating characteristics and other not so amusing.

Antivirus often get in the way of new files that are created and rendered my use of SVN to a true nightmare.

Hence, I needed to remove SVN from one of the projects that I'm working and there is no single straight forward way of doing this from the coding IDE.

Looking around for advice I see how many people recommend just removing all files starting with ".SVN". However, this is a lot of manual work when the project is of medium dimension.

I've tried it once but it's too time consuming. So, looking around the web I found a nifty way of cleaning SVN from a project.

Create a .bat file, place it on the root of your project and place the following code inside:

FOR /F "tokens=*" %%G IN ('DIR /B /AD /S *.svn*') DO RMDIR /S /Q "%%G"

I take no credit for the snippet as it came from this page: http://www.sean-barton.co.uk/2009/07/how-to-recursively-remove-svn-directories/

Hope it helps if you ever run into the same nuisance.

Getting real, the software development model

Found a very nice description of a software development model called "getting real". In a world where processes become the salvation to achieve efficient results in complex systems, we find a group with a different twist of perspective.

They advice that when competing, one should scale down the features instead of scaling up and avoid common decisions that are made too early in the project development and eventually lead to a result that nobody is happy about.

I can't say that I agree with all their comments, but in good truth should admit that our development of WinBuilder followed a good part of their guidelines to thrive.

Here's one of the things you might find inside their pages:

Be An Executioner

It's so funny when I hear people being so protective of ideas. (People who want me to sign an nda to tell me the simplest idea.)

To me, ideas are worth nothing unless executed. They are just a multiplier. Execution is worth millions.

Explanation:

• Awful idea = -1
• Weak idea = 1
• So-so idea = 5
• Good idea = 10
• Great idea = 15
• Brilliant idea = 20

• No execution = $1
• Weak execution = $1000
• So-so execution = $10,000
• Good execution = $100,000
• Great execution = $1,000,000
• Brilliant execution = $10,000,000

To make a business, you need to multiply the two.

The most brilliant idea, with no execution, is worth $20. The most brilliant idea takes great execution to be worth $20,000,000.

That's why I don't want to hear people's ideas. I'm not interested until I see their execution.

—Derek Sivers, president and programmer, CD Baby and HostBaby

You can read about them here: http://gettingreal.37signals.com/toc.php

:)

Partnership with 360Amigo

We've become the official discussion forum of products released by 360amigo a company that recently began releasing their tools to the public such as a System Speedup, a free tool for home users.

Their forum section is hosted at VirusRemoval.pro right next to the discussion forum of Ninja pendisk - http://virusremoval.pro/forumdisplay.php?fid=12

At VirusRemoval we sometimes receive invitations from security companies to test a given security suite. As a house policy on these cases, we only test and help products that are at minimum free for home users.

One of these security companies introduced us to 360amigo and the dev team behind it are good guys, so we decided to help them and bring some visibility to their free tool while also bringing some diversity to our forum discussions. A win-win situation for both ends.

You can visit the company site at http://360amigo.com and let us know at VirusRemoval your opinion.

:)

http://ous.in acquired

It costed some whooping 8 dollars to acquire this short domain that allows setting up domain hacks like http://fabul.ous.in, http://danger.ous.in or even marvel.ous.in amongst any others that you might imagine.

Despite having no idea of use in sight for "ous" at the moment, it was a good opportunity at a reasonable price. Good enough to compose hundreds of common words for domain hacks in English language at future projects.

This kind of reminds me of the Delicious website that was initially using http://del.icio.us before it was acquired by Yahoo! Inc. Funny enough that Delicious itself was founded by a Carnegie Mellon student, Joshua Schachter.

But back to earth, if you have any suggestions regarding how to put this URL into some good use, then do let me know!

:)

Boot Land keeps rising up the charts.

I'm really proud in seeing how well Boot Land is achieving good results in terms of popularity and overall ranking on a global scale.

Over these past few days we've kept on moving up and surpassing a giant site like MSFN, reaching a position on the top 30 000 sites around the globe.

30k is certainly far from reaching the top 1000 but considering the fact that our community does not deal with mainstream topics like fashion, movies or even generic PC support then I would say that things are looking bright on our arena when comparing to other sites in a similar specialty field.

Congratulations are due to the Boot Land community!

:)

New laptop - Toshiba R630

Lo and behold, I'm back on Windows!

Yesterday I've got myself a brand new machine that comes native with Windows 7 x64.

Guess this concludes a cycle that began two years ago when I removed Vista and installed Ubuntu to use it as a full time replacement desktop OS -

http://nunobrito1981.blogspot.com/2008/06/bye-bye-vista-say-hello-to-ubuntu.html

At that time, Vista was a serious nuissance and moving to Ubuntu was a real blessing to prevent me from getting bald in early age.

Now Windows 7 came. Lots of mistakes learned the hard way, hardware continuously evolving and even more experience on my side as I've been using Ubuntu and Mac OSX during this interreign.

Won't spend time talking about the good or bad things on either side but in a true sense can also say that trying to use each one of them for a certain period of time is certainly an enriching experience.

Now, I'm back on my roots and ready to have fun.

Choosing a new machine wasn't easy. So I've asked for opinion to a hardline no-nonsense structural engineer that follows the laptop trends. He needs to work intensively with Autocad, probably the most resource-hungry application in the world, so, who better to ask for an opinion? :)

This guy also happens to be my younger brother and it is interesting to hear his thoughts on new technology.

He recommended the Toshiba R630, a laptop that judging by the pictures that I saw online was probably one of the ugliest and un-sexiest machines on the market.

However, the machine is in fact a hidden gem when you read between the lines and compare it against other laptops.

It comes with an i5 processor (with 4 Intel x64 CPU cores), 4Gb of RAM, battery runs up to 8 hours on economic mode and weights some stunning 1.5 kilograms packed with all that processing power.

And when looking at the machine on real life, it is very discrete and small sized.

There's no fluff on the laptop. The display is not glossy, there are no dummy buttons and it goes against the current trend of PC manufacturers to look like a cheap copy of the MacBooks.

It's a real PC and I'm happy with it.

:)

Snippet: Fetching a regular expression

I'm including a small handy snippet to use regular expressions in your Java code.

Regular expressions allow to save time when in need of retrieving a very specific portion of text within a string. On this case, I've used to gather text from an HTML page.

/**
* Gets a string value from laptop characteristics based on a given pattern.
* A Matcher object is used internally.
*
* @param source string containing the text to be parsed
* @param reg regular expression pattern to use
* @param group index of one of the groups found by the pattern
* @return String containing the found pattern, or null otherwise
*/
private String findRegEx(String source, String reg, int group) {
String out = null;

Pattern p = Pattern.compile(reg); // Prepare the search pattern.
Matcher matcher = p.matcher(source); // Retrieve our items.

if (matcher.find()) {
try {
out = matcher.group(group);
} catch (Exception e) {}
}

return out;
}

You can use a regular expression simulator as the one available at http://gskinner.com/RegExr/ to test your regular expressions and also change the available templates over there.

:)

WinBuilder featured on C'T 2010

WinBuilder got featured (again) on the German C'T Computer magazine.

This was three months ago but only noticed it while looking on the visitor log at Boot Land.

The good guys from C'T are making available online a screenshot of the pages where the article is mentioned. You can see it in more detail at this link:

http://www.heise.de/ct/inhalt/2010/06/172/

And below are two screenshots of the magazine pages:

Guess my goal for 2011 is learning German to properly read the article as intended.

:)

Using the URL shortner from google

Google has made available their own google shortner service at http://goo.gl

They require that you install the google toolbar to use the service or you can use the tool made available by Alexandre Gaigalas at http://gaigalas.net/lab/

Using their service gives some sense of stability, I was a fan of another service at http://tr.im but when it become inactive, so went inactive all my URL's from their site.

Hope you enjoy this tip.

:)

ipkr.net sold to game developer

Last week I've sold one of the domains on my portfolio - http://ipkr.net to a game developer from Ubisoft.

It was a good trade for both ends, he ended up with a good and short domain for his project and I wouldn't really give any proper use to this domain.

All that is left is wishing the new owner good luck and best wishes of success.

:)

HTTP servers for Java 5

At my project it is necessary to ensure that each client can also become a HTTP server on their own so that they can communicate with other clients.

One would think that this task would be easier using JMS or any other communication protocols like XMPP.

However, I'm assuming a worst case scenario where the proxy for a LAN only allows connections from port 80 to the outside world and even then, checks all packages to ensure that the content of each message is real HTTP content.

I'm also assuming that the users of the client have no administrative permissions and no power to allow ports to be open or controlled by applications with guest permissions.

So, port 80 is a nice way to communicate since LAN proxies often let this door open but we still can't move past the administrative permissions required to control port 80 so I'm using for the moment 8080 as an alternative.

I've considered several communication protocols over the past two weeks, I've lost so much time looking around that I became a bit disappointed at some time. Much of what you find in Java nowadays is targeted to enterprise applications and there are good reasons why whenever someone refers to "enterprise" it might just seems like another synonym for "slow", "fat" and "octopus" to come in mind.

Let's try to change this picture then.

As HTTP server, I've abandoned the option of JMS and went forward onto plain HTTP interpretation back and forth of HTML messages (possibly marshaled with XML instructions).

Looking for HTTP servers, I've discovered that Java 6 comes already built-in with an HTTP server (link), however, we can only use Java 5 as the minimum supported java so I went looking for other projects. Found quite many of them but my favorite was this one: nanoHTTPD.

It is self-contained inside a single Java file (sized in 24kb) and brings all the basic support for exchanging pages back and forth.

As a wishful thinking, it would be nice to use the servlet power since the application is intended to be flexible and allows plugins to be integrated but I'm running out of time and other priorities need to be meet on time.

Nevertheless, here is a list of other small sized web servers in plain java that you might be interested in taking a look:

Jibble - http://www.jibble.org/jibblewebserver.php (small sized)
WikiWebServer - http://www.wikiwebserver.org (user editable)
TJWs - http://tjws.sourceforge.net (requires 7beee dependency to build)
WinStone - http://winstone.sourceforge.net (Servlet, looks professional, multiple hosts, lite version is 170Kb)

I've stumbled at an interesting article about design in a distributed computing environment.

Looking at the past, it does help to prevent some (common) design errors in the future and it sure is good to keep them in mind (regardless of how many times you hear them..)

KISS. Keep it (the design) simple and stupid. Complex systems tend to fail. They are hard to tune. They tend not to scale as well. They require smarter people to keep the wheels on the road. In short, they are a pain in the you-know-what. Conversely, simple systems tend to be easy to tune and debug and tend to fail less and scale better and are usually easier to operate. This isn’t news. As I’ve argued before, spreadsheets and SQL and PHP all succeeded precisely because they are simple and stupid—and forgiving. Interestingly, standards bodies tend to keep working on standards long after they should have been frozen. They forget these lessons and add a million bells and whistles that would, if adopted, undoubtedly cause the systems to fail. Luckily this doesn’t happen because by then there is a large body of installed code (and even hardware) out there that assumes the simpler spec and cannot handle the new bells and whistles. Therefore, no one uses them and we are all protected.

Hope you enjoy the reading, you can grab the full article here:
http://queue.acm.org/detail.cfm?id=1103833

:)

Vacations!

And hence start the summer vacations!

I'm really happy that they have finally arrived. Was getting tired of this last semester and some time out will surely be well used to refill batteries before the final semester begins.

Just wish it had gone better. At least I'm proud to say that I've learned quite a lot of new things throughout these 12 weeks. Some were good, some were bad and all of them were important to my life in one way or another.

The most memorable detail is the fact that I no longer stay away from Java. I've always avoided it, don't like eclipse, it's slow as heck compared to native binaries building a user interface is a real shame when compared to compilers for Windows that have been available for almost a decade now.

However, it does bring some fancy new advantages available nowhere else. We can indeed code once and run everywhere if things are done right. The code that runs on a desktop can also run from a server on the command line.

And.. using Netbeans as an IDE was indeed a nice touch of fresh air to escape a bulky, buggy and slow code editor.

Another nice memory from this semester is the independent study. Not only I was able to do a project from a professional perspective as it was also fun and something that I'll continue working through the next months.

For the vacations, many other projects await. Time will pass really quickly and I also want to rest. From here up to December things will not get simpler.

Luckily, from a financial perspective I'm finally at ease. Not only have I managed to secure my full payment of the MSE degree and survive the final months, as I'm also increasing my current savings and doing investments for the future.

Things haven't been easy but they're moving forward.

Avoiding stackOverflow errors in Java

I have coded a method in Java that will recursively iterate through all folders and respective sub-folders on disk.

The code seemed to work as expected, but whenever reaching a certain folder it would simply thrown an exception error and complain about stackOverflow.

Now, what is this stack overflow all about? Googling around it seems to occur whenever you enter into an endless loop situation.

This occurred while crawling sub-folders, so even thought my hard drive is quite filled up, it's not exactly filed with folders up to infinity.

So, what is wrong on this picture?

It turns out that this method is vulnerable to badly formed dynamic links. Meaning that whenever a link is found that points to a folder on lower level - it would just re-bounce back to that lower sub-folder and then loop back again ad eternum.

I've lost plenty of time trying to avoid dynamic links from being crawled but to no avail. Also reached the point of calling the absolute path of dynamic links and indexing each path on a database to check one by one if they had already been called to avoid these annoying loops (losing a lot of performance in the process).

Fortunately, found the solution to this riddle on this website - http://leepoint.net/notes-java/io/10file/20recursivelist.html

Albeit being a simplistic code, it provides a very efficient way to deal with the "symbolic-link-limbo". All you need to is define a depth level. They mention 20 as the default value and I applied the same concept on my code.

The second piece of the puzzle is verifying that each new directory that you want to crawl matches in terms of absolute path and canonical path. I've used the code from the following page as inspiration: http://www.idiom.com/~zilla/Xfiles/javasymlinks.html

After applying these changes, the code WORKED LIKE A CHARM!

Now the same method is indexing all folders up to 20 levels of depth and working as expected without seeing any more stack overflow messages. Under a MacBookPro it can index over 600 000 files under 16 minutes using less than 10Mb of heap space in RAM.

:)

My first Google engine application

Yesterday I've created my first application using Google App Engine.

I've been thinking about the possibilities of using a cloud computing environment for quite some time now. Google brings the advantage that you can use their services for free up to a given threshold of resources.

When I first heard about google app engine, it was some years ago and it was only supported on Python. At the time, phyton didn't exactly rock my world so I just walked away and kept on doing other things.

A few months ago, Java support was announced. My wild guess is that Phyton was not exactly a popular choice amongst many developers but Java is a whole different case.

So, I went on with my trial of their (free) service.

The Java environment that is officially supported by Google is based on the Eclipse IDE, since my preferences on coding fall more on the NetBeans side of life, I've "googled" for this alternative.

It turns out to be a quite simple process of configuration.

The first step is downloading and unpacking the Google App Engine SDK to some folder inside your disk - http://code.google.com/intl/en/appengine/downloads.html

No fancy install involved, just unzip and place it somewhere nice.

-------------------

Then, we open NetBeans. Google support is added as a plugin for the NetBeans IDE. There is a very good page that details these steps: http://kenai.com/projects/nbappengine/pages/NBInstall

When configuring the google plugin, I didn't selected the Hibernate Support and didn't seemed to matter much in either case.

After all said and done, you're only missing to add google app engine as a server inside your environment and this is also a simple task.

Click on the "Services" tab (that is next to "Projects" and "Files" on the right side of the IDE) and right click on the "Servers" item.

All that is missing is add a new server - select the "Google App Engine".

---------------

You should now have everything set up.

To start a web project, create a new project and select "Java Web" --> "Web Application". It will create you a template project that will be saying "Hello world!"

When you're ready to publish the project on the cloud computing environment, just right click on the project and choose "Deploy to google app engine".


Don't forget that you need to have a project already configured on the side of the google app website. After this you should be able to visit your URL and try out the results.

You can also view a full video of all these steps at the following location: http://kenai.com/projects/nbappengine/pages/Home

---------

Hope you have fun with cloud computing, this is a really simple and interesting concept at a very affordable cost.

:)

Implementing a customized Hot folder tracking with Java 1.5

One of the features required for the new project is to track changes made on specific folders.

This is the commonly called "hot folder". In the newer java versions this feature already comes implemented using the OS API whenever available.

On my case, being stuck with Java 1.5 there is not other built-in option other than pooling a folder and check for changes.

However, just checking one folder and not going deeper is still kind of limited so we need to stretch this concept further and use the imagination to check other folders without draining away all of the memory available for the java application nor hogging the CPU resources.

This was a nice challenge, used some of the learnings from the Models of Software Systems class with Pedro Bizarro, guess he'd be proud to see a finite state machine diagram being used to describe this concept. It was indeed a good tool to work out the details before moving to the implementation.

I include a small screenshot of the sketch, some changes were made on the final implementation. It's working really fast and indexing all sub-folders at blazing speed with resort to a database to save the precious RAM memory.




Let's move onto the next challenge! :)

Surviving to java.lang.OutOfMemoryError: Java heap space

Memory leaks happen.

You might be a neat and tidy person that tries to ensure that no waste is made since the start of your project coding but leaks happen.

I fell into this trap.

Was feeling nice and comfortable believing that all my code was behaving nice and using a database was keeping the memory consumption low.

Everything was working perfectly when working with a few hundred files.

However, when dealing with little over 50 thousand files the picture turned out to be grey and the evil java.lang.OutOfMemoryError: Java heap space appears.

I tried my best to pinpoint the evil doers but I couldn't really point my finger at any guilty party since none seemed to blame.

Looking for solutions, most people recommend increasing the size of the virtual machine to support the stress. But as mentioned on the opcode website: http://blogs.opcodesolutions.com/roller/java/entry/solve_java_lang_outofmemoryerror_java, adding more memory only hides the problem, doesn't really bring a scalable solution.

That blog post in particular turned out to be very useful. A good approach is using profiling tools that "police" my application and see how it behaves and which parts are not behaving nicely.

NetBeans is remarkable in that sense.

It already comes with a profiling tool that is built-in and quite simple to use.

Just click on "Profile" --> "Main Project" and follow the directions.

It will launch your project and allow you to track what is really going on underneath the hood while the program is running.

Far better than following your program from the external task manager.

I'm attaching a screenshot so that you can see how it looks like:


My application was bursting at 100Mb, with profiling I was able to track down the reason of the leak: the log entries were consuming too many resources.

Disabling the log entries you can see on the screenshot how it goes well above the previous limit using less than 10Mb of memory to index 100 000 files.

Still a lot of things to improve on my code but this nifty tool sure helps to make it possible.

:)

Java: Getting the localized path to the user's Desktop under Windows

Getting the path to the user's desktop under Windows is tricky if you're not using Java 1.6 and above.

Looking for a solution around the web, I see many people recommending to use a wrap around the Win32 API using SHGetSpecialFolderPath or reading the value directly from the host registry system as suggested here: http://stackoverflow.com/questions/1080634/how-to-get-the-desktop-path-in-java

But I'm not very fond of these solutions and found one that is simple and seems to work fairly well although it doesn't seem to be documented (yet).

Just try this snippet under a Windows machine:

FileSystemView filesys = FileSystemView.getFileSystemView();
File[] roots = filesys.getRoots();
filesys.getHomeDirectory()

And you should be able to get the desktop folder as expected. Under my language, this folder is called "Ambiente de trabalho" so it worked like a charm.

This tip comes from Russ Bradberry: http://stackoverflow.com/questions/570401/in-java-under-windows-how-do-i-find-a-redirected-desktop-folder

:)

Reading the file version from Windows EXE and DLL files.

One of my goals for this week was reading the version of Windows executable files. The code should be implemented in Java and avoid resorting to any native calls.

So, a solution that seemed simple and straightforward would be reading directly the PE header from these files and read the version number for our use.

As many things in life, it's easier said than done.

The troubles started with the code required to handle binary files. I'm using Java and had none of my trustworthy code from previous projects to read files using specific x86 sizes for DWORD, WORD and Unicode strings with a limited size.

To my rescue, I've found the nifty binary file class from Jeff Heaton: http://www.heatonresearch.com/articles/22/page2.html

It's simple and perfect. Far better than the code I used in my previous projects. To handle Unicode strings I read each byte of the Unicode string onto a byte buffer and then use a Java command to convert it properly:

Grabbing the byte sequence of the string:

rgbData = new byte[Data];
for (int i = 0; i < rgbData.length; i++){
rgbData[i] = (byte) bin.readByte();
if ((rgbData[i]==0)&&(rgbData[i-1]==0))
break;
}

Converting to plain string:

String output = new String(rgbData, "UTF-8");

--------------------------

But my biggest trouble was the fact that the file version was not kept inside the PE header itself. The file version for DLL, EXE, OCX, DRV, SCR and similar files is kept inside a resource on the file. (Thanks to TheK on boot-land for helping me sort this detail: http://www.boot-land.net/forums/index.php?showtopic=11890)

So, besides implementing the PE header reading part, it was also necessary to implement all the logic to correctly interpret resources inside executables.

For my luck, this format is extensively documented around the Internet and even MS itself has released official documentation that explains (to some extent) how the structures should be read.

Nevertheless, it took me far longer than initially expected. I had planned for a full day of work and ended up working 3 days to achieve this goal. The code itself is not optimized for speed but for the moment it will suffice the needs of the prototype.

I've tested both with DLL's from the Windows kernel and custom executables from other compilers that had inclusively been modified with UPX - the exe compressor.

It was a good adventure. I've learned far more than what I originally knew about the binary format of windows executable files and this added knowledge might certainly open the "window" for other adventures in the future.

:)

Making Java apps look good in Mac OSX

Mac OS comes bundled with Java already installed by default and strives to make the user interface easy for users.

However, for those that like coding Java applications it might seem like a daunting task to make it look user friendly. If we were coding a platform specific application, we'd just select an icon to include at runtime and be done with it.

But in java - it's just too complicated and most pages about this topic that I stumbled upon were not really shedding much light on the matter.

Eventually, I discovered that Mac OS by default also includes some nifty tools to solve this matter.

There is a life-saver application called "Jar Bundler" and you can find it inside the /Developer/Applications/Utilities folder within the Mac OS. I've found it on this page: http://developer.apple.com/mac/library/documentation/Java/Conceptual/Jar_Bundler/Packaging/Packaging.html

--------------

Using this tool is a snap, open it up, select the Jar file that you intend to run and select an icon.

Before clicking on "Create application", you will need to choose a folder that will serve as base for your application. My advice is to pick an empty folder inside the desktop.

-------------

Other tweaks:
You can click on the "Set working directory to inside application package" to ensure that any files that your jar creates are kept inside the package (to keep it nice and tidy).

The icon format used by this tool is .icns but don't worry, you can use the online service at http://iconverticons.com/ to convert the file without any pain. If you want to make your image transparent, just open it on gimp, click on "Colors" --> "Colors to Alpha" and then "Ok". If you save the image as .png then it will be lossless and preserve transparency.

That's all. Hope this tip help others creating good looking apps with Java for OSX.

:)

http://msubuntu.com

Over the last years I've been a sort of doom's day prophet in claiming that MS will eventually acquire Ubuntu sooner or later.

The reason why I claim such statement is mostly because Ubuntu is really good at what they do and the desktop/server editions just get better and better at each 6 months.

Many people think that Microsoft is the all-time enemy of Linux but in reality Microsoft was once the biggest provider of Unix operative systems even before Linux was born.

This was at the time of Xenix, a licensed Unix version that was leased to other companies for deployment in organizations. In many ways, MS contributed to make Unix better and later went to write their own history with MS-DOS and Windows (all genres).


Now, Ubuntu seems to pick on the same characteristics that made windows a platform that everyone could use for their daily work along with any other enterprise tasks.

They're quite different from any of the other Linux flavors in the sense that the focus is given on making a pleasant desktop for users and instead of pleasant users to desktop.

I can't forget the endless times when I needed to edit a pesky xorg.conf to try getting my display to work correctly. In Ubuntu you see no such thing and that can only be a good thing for those who worked with older versions of linux.

Microsoft cannot compete against an operative system that is provided at free of charge for their users but it can certainly acquire the company and profit from a leading position on the linux platforms as Canonical does at the moment.

At the moment that a MS Ubuntu version appeared, in true fact I'd see many organizations adopting linux as their default server configuration just because of the MS label posted on the box.

Inside a big organization, MS knows how to provide outstanding tools but let's face it: many times we need a simple server and paying a costly license is not a good motivator to use MS software.

Maybe this would even become a good way to refresh MS's image from a monolithic empire that is falling down against other rivals such as Apple and Google to an open company that is embracing the future and human innovation.

--------------

So, I'm going one step further with my prophecy and acquired the domain http://msubuntu.com to post my thoughts and hopes for this future to occur one day.

Perhaps more people out there also think this might become a real scenario some day and join me there, would be nice to see this happen.

Crossing my fingers..

:)

yEd - a hidden gem for those who need a simple and free diagram editor

I've been a long time fan of Dia.

Both the Windows and Ubuntu version worked good enough to suffice my diagram needs.

I just turn it on and place all the diagrams into position to later paste them onto any document. Some people like Visio better and I'm sure it is, but my interest was in using a freeware tool that didn't required a license for something that I deemed as very simple.

Well.. there is no Dia for Mac. There is an expensive replacement for Visio on OSX but I'm still interested in good and free solutions.

Looking around the web I've stumbled on a very neat application: yEd.

This graph editor is simply perfect for my needs. Runs in Java and I could start right away for a try without installing it on my machine. The design is very intuitive and thought there are some things that might take one hour or two to get used - it is very easy to use without the need to read any sort of manual.

That's the type of program that I like, free and simple.

This editor can handle UML and a lot of goodies that someone in software development will surely enjoy.

If you're in need for a simple diagram editor, you find it here:
http://www.yworks.com/en/products_yed_about.html

:)

nuno vs the CVS plugin on eclipse

If you're an Eclipse user, you probably already noticed how CVS comes as a plugin already available on the default package.

This is nice and neat to set up a new install and grab your projects without further worries.

On my case, that was the nice type of experience that I had when installing eclipse for the first time on my work laptop, a mac.

Things went smoothly, I got online to grab the documents from the CVS and no worries whatsoever.

However, my nightmare started when after a few days I've started eclipse and the CVS would simply refuse to work for no apparent reason. I know that at this point someone would say: "there is no such thing as no apparent reason" - and in true honesty I agree but have no clues about what changed.

This issue started in last February.

I've tried to solve this annoyance in numerous ways, first by uninstalling eclipse and reinstalling, then I tried the cocoa version of eclipse to see if it work differently and finally resorted to the world wide web.. to no avail on this case.

Using eclipse started to become a real nuisance. Tried asking some opinions to experts but none would come to the rescue, the most common advice: use a PC.

It just seemed that I couldn't get eclipse back to factory defaults: darn mac.

Well.. I'd be happy in using an operative system that would make my life simple but I'm also too stubborn by nature to quit from my intent of solving this trouble.

From February to this part I've reached a stalemate in my quest. I managed to get CVS access from Eclipse but the downside was the fact that I was using a fork from the eclipse project called "EasyEclipse" (http://www.easyeclipse.org)

It was indeed "easy" to use but it was also seriously outdated from the current stable version of Eclipse. This week become problematic to use an older Eclipse version because my team will all be using the stable version to ensure that everyone has the same environment.

Wish I could complain but it was guy who wrote the default environment documentation and there was simply no other way around other than facing my issue with CVS once for good.

And today: rejoice!

I've discovered how to solve this CVS nuisance. A lot of people know how to install plugins from within Eclipse. But what few people probably know is how we can also uninstall them just as well.

And uninstall I did..

I'll share this little secret: Click "Help" -> "Install new software" -> "Already installed" -> select "Eclipse CVS client" -> click "Uninstall"

After this, just restart eclipse and install CVS again from the default eclipse repositories.

So simple to fix and so many headaches to find the solution...

:)

virusremoval.pro begins

Yesterday I've announced the virus removal community forum across the good people of boot land.

The feedback response was surprisingly good. The site had been started little over four weeks ago and counted with an average of 100 daily visitors, yesterday we've peaked at over 1200 unique visitors.


This kind of response is truly motivating. The quality level of the ongoing discussions is also very good, there is a lot to be learned and we do have the conditions to give malware authors a really bad headache.

:)

Email robustness

Ever since I've first got my own personal domain as http://nunobrito.eu some years ago it became possible to use my official email address as mail@nunobrito.eu

Instead of having several mailboxes I've decided to concentrate them all at my gmail account using email redirection.

It's simple, worked well.

However, the server where my domain is hosted would often become offline for several days in a row due to the sheer amount of people trying to access the other services hosted there.

This brought me for the first time to a fragile situation because I would no longer be receiving emails while the server remained offline.

I've devised a manner to provide email robustness at no extra costs and without requiring a server.

It's very simple.

- The first step is registering your domain with goddady. If you've already registered with their service then it's fine, otherwise I'd seriously recommend moving to this particular registrar.

- Then register for google apps standard edition: http://www.google.com/apps/intl/en/group/index.html
Input your domain as the domain that will host the apps, don't worry because this won't mix with your site in case you already have it developed.

- After registering your domain with google apps, look on this page regarding how to configure the mx records at godaddy: http://www.google.com/support/a/bin/answer.py?hl=en&answer=33353

But my favorite part is this automatic tool to do it quickly: https://www.godaddy.com/gdshop/google/gmail_login.asp

The above link will automatically configure your domain to use gmail as the mail service.

- At this point the email is handled by google apps, let's manage your site:
https://www.google.com/a/cpanel/example.com (change example.com with your domain)

- After login, click on "Email"

- Click on "Change URL" and select the URL that is hosted by google so that you don't need to use your own domain to log onto the webmail service.

- Click "Email addresses" and then add a user.

- Select a user listed there and add "nicknames" to the user. This means the possible email addresses that will be used to route the messages.

- Log onto the webmail service and set the routing of all messages onto anywhere you want, just like you would do with a normal gmail account.

- Done!

You are now routing all your messages sent to your official email address onto anywhere else you need.

Good luck.

--------------------

Now my official email is robust and no longer depending on any server.

:)

Back to my roots

This is my final week in Pittsburgh. I've completed the second semester just like all other students from the MSE program.

Pittsburgh was really nice but now it's time to return onto Portugal where the remaining two semesters will last until December.

I should be happy now. The most difficult times should have passed and things from here forward should actually be more fun: I'll be coding and creating the studio project that we have been planning for so such a long time now.

However, I'm not happy. Returning to Portugal means a direct confrontation with the army in my country. From my relatives I can only gather some whispers and rumors. Even my lawyer is mute as the stormy days approach the road ahead.

Upon the idea of returning to my roots I feel similar to a small kid returning home after doing something wrong and afraid of a reprimand from his father.

And in fact, the army is still like family to me. They helped me as a teenager when the future was not promising, providing the tools, resources and discipline necessary to survive.



Now it's time to discover the world outside my "adopted family" even if it means facing the anger or disappointment of those that expected me to continue in the same career.

But this path is not easy. Usually I don't get tired, I might get exhaust and need to sleep for a good night or even get a good coffee to start a fresh day, but right now I'm just getting tired of all these obstacles.

Guess I need to remind myself that we all have a purpose and I'm still defending what I believe to be correct.

I might not be happy about the future but I'm certainly happy to have reached this far in my goals for life.

Even if I could go back to the past and be faced with the same decisions, I'd still do them. Wouldn't trade the knowledge, people and memories for anything less than a chance of truly start living with freedom of choice.

And having a choice really makes me smile.

:)

Can our legacy outlive us?

Thought this is sort of a tabu matter, no software developer or webmaster likes to wonder about what will happen when they are no longer around to assure the survival of their legacy works.

Unfortunately, I'm already forced to think about these matters due to certain attributes and choices of my life that might shorten my available time span to take care of them.



Over the years, I've done software and helped raising websites with people that now depend on my direct support to keep things going on daily basis.

Every now and then a new situation appears that needs to be solved. Today one of the subdomains was reported as hosting a malicious page, meaning that someone from the outside managed to break our perimeter defenses and use our server for their malicious purposes.

Our server was automatically shutdown by the hosting provider until I could talk with them and remove the malicious page.

I love keeping things tidy and running well but I'm certain that when I'm no longer around to assure this type of service, few other guys will be available or willing to carry forward the needed support to provide the things that we achieved over the years.


It would be unfair to so many that trusted in us as guardians of their knowledge to fail in this task. Websites like boot-land.net and projects for winbuilder deserve to endure long after the initial authors and patricians are long gone.

My own memories, hosted at nunobrito.eu are also depending on my own administration to be available.

This is clearly a solution that cannot last for centuries to come. One idea would be moving to the platform of a bigger provider like google, yahoo, microsoft, flickr or facebook.

But none of them work will likely work like we need. Anything hosted on their platforms is outside our own management and can be lost at any given moment.

How to solve this?

Thought a complete automation and debugging of issues that occur is never possible to achieve, it is possible to prepare the path and mechanisms to allow others to follow our steps and also ensure that this process is simple to keep things moving.

--------------------

The first worries are costs that are divided between hosting costs and domain renewals.

To support these costs, the publicity gathered from the bigger sites is currently enough to keep the site self-sufficient. Revenue from publicity is directly deposited on a specific paypal account and is driven from two separate advertising channels to ensure redundancy.


The first step to solve the matter of domain renewal is concentrating all domains on the same registrar (I've chosen godaddy as they are the leading provider of this service). This is not as simple as it seems, the most important domains such as winbuilder.net and boot-land.net are hosted on dominios.pt that are aggressive about domain transfers. Will try to solve this.


The second step is automating the renewal process. Currently, as paypal is the standard web payment tool, I can automate the automatic withdraw of funds to keep registrars happy.

Once the domain renewals are solved I will focus on the hosting plans. Currently we have two servers. One of them is directly sponsored by R1Soft. Unfortunately I'm unable to get in contact with them to transfer the sponsoring to the second server that has far better conditions than the first.

At the current state we are having a server that is expensive (110 USD/month) and barely used along with the new server that costs around 140 USD/month.

The most problematic situation is the new server. 140 USD/month cannot be payed from paypal. This bill needs to be settled with a credit card or direct bank transfer from an account located in Germany. The problem is that credit cards have a limit date on years that are valid. This is not a solution that will last for long since we'll always be vulnerable to a lack of payments on the bank account that is used for this purpose.

Contacting the hosting company to support paypal as revealed itself a fruitless effort. They are afraid of paypal and will only keep the traditional payments.

There's a saying in Portugal: "If maome doesn't go to the mountain then the mountain will go to maome". Guess the next logical step is opening a bank account in Germany. Being an european citizen and having good friends in Germany it shouldn't be too difficult thought I'm not yet sure if it is possible.

From this point it would be needed to ensure that paypal transfers enough revenue onto the german account at each month.

One thing is certain, without assuring hosting bills to be paid flawlessly our projects will be vulnerable to extinction.

--------------------------------------

Assuring a good flow between expenses and revenues is essential. But unfortunately this isn't something that a software engineer can code as a program to be compiled and executed.

We will always need a human operator to ensure that things keep running even when something unexpected occurs. So, the challenge is preparing the management of our financial balance to someone of trust. But how can we asses if someone is truly trustworthy for this task?

Too many open variables at this time to properly solve this challenge, guess we'll need to use a credit card for the next 2~3 years and renew this value at each year until a new hosting provider is found or otherwise really open a german account to get this settled.

----------------------------


The second point is server administration. How and where can we find people capable of managing our servers?

The technology is fairly standard but none of the servers follows traditional implementations, they were customized for speed and performance when subdued to heavy loads or special behaviors by winbuilder projects.

I know how to get things working, how can others learn to do the same?

It would be nice to completely outsource winbuilder projects to run from other service providers and ease our administration tasks but who would be willing to support terabytes of demand per month and providing gigabytes of space on their disks?


--------------------------------


Things are not easy for us but none of the folks at boot-land quit when faced with challenges nor will I quit from finding a decent solution while I'm still here.



When considering all the adversities required to ensure that our legacy can outlive us in the Internet, most people would say that we are simply outnumbered.



And perhaps they are right. The odds of ensuring that so many different services with such complex characteristics survive the endurance of time is simply nuts.

But having strong odds against this endeavor doesn't make it impossible to achieve. With a good strategy we might just be able to use our own resources to reach the expected results.


There is no way of supporting the costs of renting the services from a cloud computing hosting or hiring specialized support personnel. Given all these constraints we might as well dive into building our own robust cloud and administration solution to allow our legacy survive in years to come.

We wouldn't just be reinventing the wheel, we'd be redefining a whole new vehicle to take us where we need to go.

Perhaps this is indeed nuts but a worthy challenge as nobody else seems (yet) be considering this type of issue and we do have a tradition of moving ahead of the game..

:)

Are we romans?

For a long time, Europe lived under the rules of the Roman republic and later, the Roman empire.

Throughout this time, roads were built and civilization was brought to places where the people previously had habits not too distant from those on the ice age.

Roman civilization meant education, health and wealth to those who embraced roman culture as a way of exchanging goods and traveling in security to other provinces.

However, the West Roman empire where France, Germany, Portugal, Spain and Italy exist today was progressively torn apart across the centuries by successive invasions of barbarians from the Northern parts of Europe until the roman empire in the west ceased to exist, marking the start of the medieval ages as we know them.


This remark in history is a true contrasense. If at one hand we had such an advanced culture in terms of technology, culture and organization - what went wrong to let the barbarians took over?

The explanation is plain brute force from the barbarians and faulty roman policies in terms of administration.

We see the roman history filled with heroes that shaped the republic and empire to success and falls across the ages, yet, even the barbarians admired the roman culture and attempted to replicate the same level of culture to no avail most times.


What can we learn as a lesson of history?

Civilized cultures are not prepared enough to survive against brute force threats from the outside or even weak management policies.

We see the same issues plaguing computer systems nowadays. Computer users attempt to secure their own machines from outside threats but are unable (or unwilling) to keep them tidy with effective security policies.

The barbarians are today's malware authors that use conquered computers as slaves to power monstrously sized botnets that serve as weapons to attack companies and organizations across the globe.

This is the reality today. We have barbarians and we have civilized citizens trying to work. But we still have little means to fight back the threats and propose truly efficient methods to fight them back, instead, we rely in defense and few actions are taken to go after the wrong doers.

Looking to where we are headed, will we let history repeat itself with the fall of a civilized Internet?

I think we should learn from the lessons of the past that we cannot close our eyes to the enemies at the border and expect that our own organization will be able of solving these menaces some day.

So, I would like to propose a roman-like concept of cooperative defense for the internet world.

Back in the old days we would see farmers picking up anything at their hand to defend their lands. This reality is not far from today where computer users install antivirus at their machines to keep them protected.

However, would it sound reasonable for you that a farmer is truly capable of defending himself against an aggressive tribe of barbarians that are professional hackers?

Something is truly wrong in this picture. Over the next months I will dedicate my effort to prove that we can indeed be romans. Prove that civilized people can fight back the attacks from barbarians and level the weights on each side of fence.

Let's begin.

Finishing up the second semester

It's been 4 months since I've arrived to the States and now it's time to go.

Pittbusrgh was very nice to me. The people, the university and the town were pleasant and welcomed me inside their traditions and culture, making me feel as part of this place instead of an outsider.

I will soon finish the second semester and head back to Portugal but things won't be easy there.

From an educational perspective I'll be working to implement our software engineering studio project and will also be doing an independent study proposal on a topic that was self-proposed.

The tough part for me will be finding a place to stay, where to study and how to manage my situation with the Portuguese army alongside with balancing my financial status to ensure that I can support all these expenses.

But things are moving forward and a lot of people are helping. I can't predict what will happen next but at least I'm happy to still be studying and getting closer to complete another step in the MSE challenge.

:)

Link exchange with MSFN

MSFN is one of the oldest and most respected sites when it comes to Microsoft discussions. It was created in 2002 with the purpose of sharing knowledge regarding how to tweak windows to work how fans intended.

Over the years it has specialized on the customization of unattended windows installations, some of the most popular tools that spawn from this forum are nLite and vLite that reached a global scale success.

Boot Land started in 2006 when MSFN was already a giant in the Windows arena. Over the years we have also grown to the point where we have become giants in our own domain.

MSFN and Boot Land have mutually exchanged links. To date, we are the only forums recommended on MSFN and we also come a long way to deserve such attention.

My personal thanks to everyone who supports the boot land community!

Virus Removal Pro

I've started a new project last week, it's a new community forum entitled "Virus Removal" that can be found at http://virusremoval.pro

Boot Land was and remains a success in terms of a community focused on the development of boot disks but I've been feeling that it is also time to expand to other themes that are also important and interesting to explore.

Funny, but I've first started creating boot disks to repair my computer after it was attacked by a virus. So, in a good sense I can say that both activities have also been part of my routine ever since I was a kid.

This new community has a lot of good things.

I've used myBB as forum software, the strong point is that it won't require any costs with licenses since it is free, also, I can add plugins in a really easy manner that doesn't require editing any file. In overall it is a very simple and straightforward software to use.

The domain itself is very easy to remember and straight to the point. Our forums at Boot Land are also being used to help kickstart the forum and bring new visitors.


Last but not least, virusremoval.pro already counts with the participation of valuable experts in this field.

-------

So, it seems we have all the right ingredients to start a good community that can actually contribute with something fresh and useful to people around the Internet.

:)

http://freeflippa.com donated to the owner of http://flippa.com

Last year I started a project meant to provide a free alternative to the services provided by Flippa, a website dedicated to "website flipping", the art of selling websites that were just created.

I've chosen the domain http://freeflippa.com and shortly after opening the doors for this new site, the owner of flippa.com approached me by email with concerns about trademark assurance.

Following the european laws, nothing would restrict me from using this specific website domain since it is not a trademark recognized in Europe or the States.

Nevertheless, I'm not the type of person that enjoys "cybersquatting" and removed any resemblances on the site that would compete against the paid services provided by flippa.com

Two days ago I've decided to clean up my portfolio of domains and remembered that I still had freeflippa. Decided to send an email to the owner of flippa and I've transferred freeflippa.com to his ownership at zero cost.

Webmaster projects are very attractive to some extent but I guess that I'm just not cut for them.

Will rather stick with software development, it was nevertheless another episode on my life to keep in memory.

:)

New server!

A new server was added to power up our sites.

The new machine is a brand new i7 server with 8 CPU cores and a whooping RAM of 24Gb running under Ubuntu x64.

The configuration was a bit more problematic than expected. The older server had all domains managed with Plesk but this platform was not supported on the new hosting and I simply couldn't afford the added cost for a plesk license at each month.

So, I've decided to try out the free alternatives to cPanel or Plesk. My attention was first tuned onto webmin and virtualmin since they allow to manage most aspects from a server quite easily.

However, the management of virtual hosts under Apache soon turned out to be an headache to solve. Some bugs occurred and then it became very difficult to get things working as expected.

Disappointed with Apache, I've turned my attention to Nginx as a web server. However, the lack of documentation and clear examples also led me to fail using it as expected.

Last but not least, came lighttpd. It's reputation for performance is well know and I decided to give it a shot. It turns out that this server has reached a very mature and stable level of performance and was only an apt-get away from being installed on the machine.

Out of the box came the support for PHP just like I needed. Adding a new virtual host is a task as simple as adding a new line on the configuration file and restarting the server.

As for speed, boot-land.net is now loading at a blazing fast speed. Lighttpd is serving static files at a really great speed on a far faster server than ever before and I'm happy to see it working well.

I still need to configure some other details. Luckily I didn't had any sites requiring mod_rewrite



Attached is a screenshot of the new server in action, feels really good to see all works being distributed amongst the 8 cores with plenty of RAM to go about.

The monthly costs for this server are really high, especially if we take into consideration that the older server was fully sponsored by R1soft for free and that now I'm supporting the new hosting costs on my own.

But I was unable to get in contact with them and the boot land sites were in dire need for better conditions.

Let the fun begin.

:)

Bring on the bots..

We've been having a lot of fun over the past two weeks in our boot land server.

It's been a long while since the last time we got offline for such a long period of time (2 years?), at the time was required for overcoming the attack that we move from a virtual shared server onto a fully dedicated server.

At the time, our growth just seemed to keep going up unexpectedly and suddenly we were having thousands of visitors from all around the world looking for boot disk solutions.

This was two years ago.

Currently.. boot land grown onto thousands of daily visitors and terabytes of monthly information being exchanged from the server.

We've been happy and busy working on the new things that appeared ever since. However, there is still one situation that is challenging our capacity to survive the test of time: bots.

Given the current position of boot land as a popular site, we are experiencing a mass wave of bot machines that visit our site and associated domains (winbuilder.net et al), causing our resources to be quickly depleted into what people would call as a DDoS. Typically, this is a obviously a good way to ensure that a competing site is sent for oblivion.

And a strong DDoS it was indeed. The server was not prepared and a massive black out got installed for 5 days in a row.

But.. we're alive. Sometimes slow at serving pages, it's true, but we're winning back our server to the bots that were sent our way.

So, if for any chances the person(s) behind this recent bot incursion wind up reading this blog post, do note one thing in particular about boot land: we're here to stay, bring on the bots.

Isn't the snow fun?

And I've arrived to Pittsburgh.

The snow is everywhere, the cold as well but life is fresh as it hasn't been for a long while.

Why? This is the first time that I actually saw real snow inside a city.


Temperatures remain below zero celsius and I've took these free days to visit the city and saw all type of things from America that I had only seen on television.

Pittsburgh is a relatively small town when compared to others, but still sized enough to host the tallest buildings that I ever seen. And while writing this post I can only think: what a simple guy, never moved away from the village before, and in many aspects, it's true: living in small towns does carry some interesting advantages that I truly love.


It's good to be here. A good time to shape up my English skills and try to see all there is to see around Pitt.

Money also flies away from your pockets at an incredible speed, life is nowhere as affordable as I'd expect and simple things like lunch, dinner and coffee are pressing a tough toll on my savings.

But well. I'm still adjusting and thinking a lot about the things back in Portugal. My employment is still an headache to solve, the savings to endure the following months are also a worry and top all this with the constant pressure to think about your work at the MSE program.

Life is amazing, I seriously wouldn't imagine how it could change so much in less than 12 months and be here today.