Posts Tagged: Programming


12
Nov 10

Software development is the newest blue collar trade

Traditionally computer science is a white color discipline, a cerebral activity beyond that of the typical trades. While not all computer scientists are software programmers, most of the things people touch on their computers and on the Internet run code that developers wrote. Developers may have worn white collars at one time, but are now more than ever better served if we dress them in blue.

Understanding what makes great software developers needs to become a top imperative or everyone’s desire to successfully leverage the developing economies of the globe will result in the next decade of disastrous implementation. We will all literally be digging out of the worst collection of computer code the world has ever seen. This is not to say programmers in developing countries are not capable of creating great code – clearly that would be too broad a generalization. What I am saying is that there is a core set of existing developers – waves one through five – that have created the software and network conscious of the world. That experience and knowledge is not easily portable locally or internationally. More needs to be done to consider the ways in which we grow developers. The fact that everyone is quick to move to emerging markets is simply exacerbating the fact that the Western world contains much of the building blocks everyone takes for granted.

There are classes of programmers that have never written the basic code to connect a web application to a database. They use any number of indirect frameworks to achieve what is a relatively straight forward activity. It may be laborious, but it also results in a development team that understands what is happening at every moment in the system. Delegate to someone else and your risk is that whatever was to be done is performed less well than if you performed the task yourself. There are plenty that will tell you it is a given since it’s the only way to scale yourself as a person. When we are talking about computer code the exposure is as great as the worst written code. Perfection is not required, but ignorance is worse.

I recently got passed the essay Shop Class as Soulcraft by Matthew B. Crawford and with a little positioning should be the playbook for America’s future – possibly, eventually, the world. Crawford does a delightful job exploring what it means to be engaged in a trade – its scarcity, importance and value.

My recent gap in blog posts reflects that my life got too busy to support the usual post – selling a house, buying a house, moving and building out a studio to compliment the changing lifestyle. I share this because it is likely the first time I took on the challenge of building something by hand that I would usually create with money. There is no shortage of cerebral activity in building and Shop Class as Soulcraft makes this point well. It is easy to liken it to software programming in that you need to understand fundamental principles – logic, algorithms, design patterns etc. This is not much different than a builder understanding material strength, stability and appropriate use. Programmers feel the same pride and satisfaction from code well-built as a trade person elegantly executing their craft. While there is a notion of mentorship and hierarchy the trades have a more structured concept around apprenticeship. This is a critical aspect acknowledging that some of the knowledge to be had is hard if not impossible to distil or consume in traditional forms. Experience efficiently encodes more information that our conscious mind processes, yet our beings embody the knowledge.

There is no shortage of computer programmers in the world, yet there is a dearth of individual and shared development experiences. We can’t expect everyone to live through the trials of personal computers or the Internet, but we do need to bridge the gap or not only will we repeat history, but we won’t have enough people to fix it all when it is broken. No different, Crawford points out that with the dwindling ranks in the trades the individual that understands how to do something with their minds and hands will become the most important person in the village. While I clearly agree for the need to embrace the world’s crafts, I believe we are facing an epidemic that must be reversed. Just as the established markets have created a pile of stinking code, failing to pass knowledge to newer generations, we extend the work to nations that have even less shared knowledge. We must apply the methods of the trades to software development or fail faster before the shared knowledge ceases to exist.

I will wear a blue collar any day since it transcends what use to imply class and embodies a healthier balance of being. Read Shop Class as Soulcraft and figure out how to help fix us before we are broken.


19
Dec 08

Winding down. Hacking PERL.

Vacations are for getting away from work and resting from the daily grind. Yet, almost everyone seems to need a week to unwind before getting to relax. My week of unwinding was productive.

During my last workweek, I was helping an employee configure Apache to simplify her life. While poking around on the development machine I noticed that the team was creating multiple WebSphere Application Server profiles. WebSphere Application Server (WAS) is IBM’s J2EE platform and it supports the ability of running multiple instances of the runtime environment without installing the product several times. This can be a very powerful feature and in the case of my team, they were simply doing it for isolation. It allows each of them to manage an instance of WAS without worrying about affecting other peoples work. While this is a development machine, apparently one of the applications it ran is less “in the works” than the others. This presented a problem, when I wanted to front-end my employee’s application (the one I just configure Apache for) with the web server.

You see, in a typical WAS setup, there is a web server that receives requests from an end user’s web browser. Web servers are extremely good at serving static content, executing CGI scripts and connecting to FastCGI applications. To hook into this layer, WAS loads a plugin that reads an XML configuration document to understand what it should do. This document, among other things, maps which URL paths should be passed on to WAS. For each WAS profile, a new plugin-cfg.xml file is generated. This makes sense, because each of these profiles is a different environment. The challenge is that the WAS plugin Apache loads can only read one XML file. So, what do you do when you want to map multiple WAS profiles to a single web server?

A few months ago, I remember the manager of the systems team we work with describing how this was a manual process and when he had more time, he would automate it. This led me to search and find a link for Lotus Connections documentation, “Mapping multiple WAS profiles to a single IBM HTTP Server,” that described the manual procedure. Using this as my guide, I updated the plugin-cfg.xml file manually and my employee was all set. It worked as expected and that was that. My last workday was the following day and yet I was compelled to automate this during my wind down week.

PERL is my programming language of choice when it comes to working with text and it fits perfectly for this task – merging two plugin-cfg.xml files. There are four XML nodes to consider when merging these files: VirtualHostsGroup, ServerCluster, UriGroup and Route. The latter three are easy because they just need to be copied “as is” into the merged XML document. VirtualHostsGroup however needs to be merged. Moreover, while the example from the Lotus Connections documentation shows a VirtualHostsGroup stanza named “default_host” and because this is XML, to do this responsibly we need to presume there could also be a VirtualHostGroup named something else. So, not only do we need to combing the children of all VirtualHostGroup nodes, we need to persist these nodes even if the named node in one XML file is not in the other.

With the help of XML::LibXML and XML::Tidy this task is simple enough. The following script takes three arguments, the file name of first XML document, the second XML document and the new XML document. It merges the second document into the first, inserting changes after the existing XML nodes. This approach ensures the file appears in a more logical human readable fashion. The formatting options available in LibXML disagreed with my aesthetics, so I added a dependency on XML::Tidy, which while cleaning up the document, makes it too neat. Oh well. The script could be used multiple times to merge several plugin-cfg.xml documents. For full disclosure, the IBM documentation for Lotus Connections mentions:

Do not complete this procedure if you are planning to add the multiple profiles to a node in a network deployment. In that case, you can define a Web server for the node and map only the node to the IBM HTTP Server.

I have not looked into what happens or what would need to happen to support multiple profiles in a network deployment. My last disclaimer is that I have tested this many times with a few edge cases. As with any software, it may have bugs, so test it on your documents before using this in a production environment. I will not be held liable for damage you create using this script. For that matter, if you plan to use it commercially, you should email me to discuss a license. Let me know if you find a bug or have suggestions.

Download wpcMerge.zip.