Posts Tagged: Software Development


12
Nov 10

Software development is the newest blue collar trade

Traditionally computer science is a white color discipline, a cerebral activity beyond that of the typical trades. While not all computer scientists are software programmers, most of the things people touch on their computers and on the Internet run code that developers wrote. Developers may have worn white collars at one time, but are now more than ever better served if we dress them in blue.

Understanding what makes great software developers needs to become a top imperative or everyone’s desire to successfully leverage the developing economies of the globe will result in the next decade of disastrous implementation. We will all literally be digging out of the worst collection of computer code the world has ever seen. This is not to say programmers in developing countries are not capable of creating great code – clearly that would be too broad a generalization. What I am saying is that there is a core set of existing developers – waves one through five – that have created the software and network conscious of the world. That experience and knowledge is not easily portable locally or internationally. More needs to be done to consider the ways in which we grow developers. The fact that everyone is quick to move to emerging markets is simply exacerbating the fact that the Western world contains much of the building blocks everyone takes for granted.

There are classes of programmers that have never written the basic code to connect a web application to a database. They use any number of indirect frameworks to achieve what is a relatively straight forward activity. It may be laborious, but it also results in a development team that understands what is happening at every moment in the system. Delegate to someone else and your risk is that whatever was to be done is performed less well than if you performed the task yourself. There are plenty that will tell you it is a given since it’s the only way to scale yourself as a person. When we are talking about computer code the exposure is as great as the worst written code. Perfection is not required, but ignorance is worse.

I recently got passed the essay Shop Class as Soulcraft by Matthew B. Crawford and with a little positioning should be the playbook for America’s future – possibly, eventually, the world. Crawford does a delightful job exploring what it means to be engaged in a trade – its scarcity, importance and value.

My recent gap in blog posts reflects that my life got too busy to support the usual post – selling a house, buying a house, moving and building out a studio to compliment the changing lifestyle. I share this because it is likely the first time I took on the challenge of building something by hand that I would usually create with money. There is no shortage of cerebral activity in building and Shop Class as Soulcraft makes this point well. It is easy to liken it to software programming in that you need to understand fundamental principles – logic, algorithms, design patterns etc. This is not much different than a builder understanding material strength, stability and appropriate use. Programmers feel the same pride and satisfaction from code well-built as a trade person elegantly executing their craft. While there is a notion of mentorship and hierarchy the trades have a more structured concept around apprenticeship. This is a critical aspect acknowledging that some of the knowledge to be had is hard if not impossible to distil or consume in traditional forms. Experience efficiently encodes more information that our conscious mind processes, yet our beings embody the knowledge.

There is no shortage of computer programmers in the world, yet there is a dearth of individual and shared development experiences. We can’t expect everyone to live through the trials of personal computers or the Internet, but we do need to bridge the gap or not only will we repeat history, but we won’t have enough people to fix it all when it is broken. No different, Crawford points out that with the dwindling ranks in the trades the individual that understands how to do something with their minds and hands will become the most important person in the village. While I clearly agree for the need to embrace the world’s crafts, I believe we are facing an epidemic that must be reversed. Just as the established markets have created a pile of stinking code, failing to pass knowledge to newer generations, we extend the work to nations that have even less shared knowledge. We must apply the methods of the trades to software development or fail faster before the shared knowledge ceases to exist.

I will wear a blue collar any day since it transcends what use to imply class and embodies a healthier balance of being. Read Shop Class as Soulcraft and figure out how to help fix us before we are broken.


19
Dec 08

Winding down. Hacking PERL.

Vacations are for getting away from work and resting from the daily grind. Yet, almost everyone seems to need a week to unwind before getting to relax. My week of unwinding was productive.

During my last workweek, I was helping an employee configure Apache to simplify her life. While poking around on the development machine I noticed that the team was creating multiple WebSphere Application Server profiles. WebSphere Application Server (WAS) is IBM’s J2EE platform and it supports the ability of running multiple instances of the runtime environment without installing the product several times. This can be a very powerful feature and in the case of my team, they were simply doing it for isolation. It allows each of them to manage an instance of WAS without worrying about affecting other peoples work. While this is a development machine, apparently one of the applications it ran is less “in the works” than the others. This presented a problem, when I wanted to front-end my employee’s application (the one I just configure Apache for) with the web server.

You see, in a typical WAS setup, there is a web server that receives requests from an end user’s web browser. Web servers are extremely good at serving static content, executing CGI scripts and connecting to FastCGI applications. To hook into this layer, WAS loads a plugin that reads an XML configuration document to understand what it should do. This document, among other things, maps which URL paths should be passed on to WAS. For each WAS profile, a new plugin-cfg.xml file is generated. This makes sense, because each of these profiles is a different environment. The challenge is that the WAS plugin Apache loads can only read one XML file. So, what do you do when you want to map multiple WAS profiles to a single web server?

A few months ago, I remember the manager of the systems team we work with describing how this was a manual process and when he had more time, he would automate it. This led me to search and find a link for Lotus Connections documentation, “Mapping multiple WAS profiles to a single IBM HTTP Server,” that described the manual procedure. Using this as my guide, I updated the plugin-cfg.xml file manually and my employee was all set. It worked as expected and that was that. My last workday was the following day and yet I was compelled to automate this during my wind down week.

PERL is my programming language of choice when it comes to working with text and it fits perfectly for this task – merging two plugin-cfg.xml files. There are four XML nodes to consider when merging these files: VirtualHostsGroup, ServerCluster, UriGroup and Route. The latter three are easy because they just need to be copied “as is” into the merged XML document. VirtualHostsGroup however needs to be merged. Moreover, while the example from the Lotus Connections documentation shows a VirtualHostsGroup stanza named “default_host” and because this is XML, to do this responsibly we need to presume there could also be a VirtualHostGroup named something else. So, not only do we need to combing the children of all VirtualHostGroup nodes, we need to persist these nodes even if the named node in one XML file is not in the other.

With the help of XML::LibXML and XML::Tidy this task is simple enough. The following script takes three arguments, the file name of first XML document, the second XML document and the new XML document. It merges the second document into the first, inserting changes after the existing XML nodes. This approach ensures the file appears in a more logical human readable fashion. The formatting options available in LibXML disagreed with my aesthetics, so I added a dependency on XML::Tidy, which while cleaning up the document, makes it too neat. Oh well. The script could be used multiple times to merge several plugin-cfg.xml documents. For full disclosure, the IBM documentation for Lotus Connections mentions:

Do not complete this procedure if you are planning to add the multiple profiles to a node in a network deployment. In that case, you can define a Web server for the node and map only the node to the IBM HTTP Server.

I have not looked into what happens or what would need to happen to support multiple profiles in a network deployment. My last disclaimer is that I have tested this many times with a few edge cases. As with any software, it may have bugs, so test it on your documents before using this in a production environment. I will not be held liable for damage you create using this script. For that matter, if you plan to use it commercially, you should email me to discuss a license. Let me know if you find a bug or have suggestions.

Download wpcMerge.zip.


10
Nov 08

Prototyping can lead to remarkable outcomes

Prototyping is something software builders leave to user experience professionals and that is completely insane. Prototyping should inhabit everyone’s being. It is an opportunity at any step of any process or creation. There are too many first drafts called final drafts, especially in software.

Regardless of your software development methodology, agile or waterfall, there is always a notation that says, “Take time to prototype.” Practitioners speed by this phase as if it were distracting from the final form. Project leaders receive pressures to deliver more, in less time and at or under budget. Where in that formula is prototyping placed at the top?

Some companies handle this by creating smaller development teams, housed in research and development or branded agile extensions of the larger delivery organization. This is where people think the prototypes come from. Does location, context or reporting structure have any bearing on how real software is?

pro⋅to⋅type [proh-tuh-tahyp]
noun, verb -typed, -typ⋅ing. –noun

  1. the original or model on which something is based or formed.
  2. someone or something that serves to illustrate the typical qualities of a class; model; exemplar: She is the prototype of a student activist.

1595–1605; < NL prōtotypon < Gk prōtótypon, n. use of neut. of prōtótypos original.

When I joined a skunk works team at IBM directly from college, everyone said they created prototypes. These prototypes were designed and delivered to scale to the entire enterprise. People called them prototypes to set expectations and help the traditional brass understand why a one-year project might be delivered in three months. Rarely was there anything approximating a prototype – well maybe the very first version.

There have been notable individuals that would have the courage to rewrite what they had coded to achieve greater aesthetic or optimal execution. It is a humbling, empowering and inspiring experience to throw out your current work in attempt to write it again better than before. This is the closest I have seen to developers creating prototypes and this is a rare occasion because we so easily rationalize how there is so little time, so little money and so much more to do.

First drafts make lame software, hence we iterate in hopes to accelerate the separation of curds and whey. In the end, this is not prototyping. Prototypes offer the opportunity to understand what is uniquely delivered by the solution, what is important to end-users and how the way the solution is built helps or hurts those two points. This is why the process often ends up being a user experience deliverable. The challenge and thus opportunity, is that maybe what is deemed important is unfounded. Maybe the feedback users provide is based on a poor articulation or misguided offering.

Prototyping in software development educates the architects, the developers and, in turn, the user experience professionals. By building relatively low investment prototypes through rapid development tooling, the notion of throwing the resulting build away seems palatable. It offers the opportunity to learn without entering the software delivery cycle – skipping the prototype is the quickest way not to have freedom to try things. Once there is focus on solution delivery, everyone creates a context that prevents the exploration of what is right. Everyone is left to struggle with making what gets delivers as good as it can be, given the circumstances.

This is all likely generalizable to many professions. Creative types work through revisions as part of their craft. Prototypes educate you, your team and others to the importance of the end direction. It forces everyone to approach the real work with purpose and limits the exposure to making poor decisions under pressure or pretense. Spending the time up front pays everyone back during and after delivery and yet it is not prioritized as such. Instead, we hope that professionals executing a good enough approach will deliver something remarkable.


5
Oct 08

What are we saying?

It is encouraging that people find analyzing data so compelling. Visualizations like the ones you can find at Digg labs can whet the appetite of almost anyone. Environments such as Many Eyes allow users to engage more directly in the dialogue of information exploration. Wordle, a tool that enables you to generate your own word clouds makes visual statements on views that go unnoticed.

Creating a Wordle visualization of your resume seems to be something people enjoy. It reflects back the author’s personal language for articulating their experience and qualifications. I wonder how many altered their documents to direct the impressions they were creating.

People seem to enjoy sharing word cloud views of the news and politics. Wordle generates beautiful pictures using word frequency; the more often a word occurs the larger it renders. This means that what you put in directly affects what Wordle can turn out. While it includes the flexibility of lower casing words, removing noise words and interactive editing if you spend any time with Wordle, you will find yourself tweaking your content.

In an attempt to practice my PHP skills, I created a simple utility to help automate some text processing prior to working with Wordle. Think of it as the presoak cycle of the word cloud creation process. While it is humble in its scope and function, it can heighten the impact of your visualization. Check out Wordle Presoak if this is the kind of thing you are in to.

This composition below presents variations using Wordle Presoak on the same text pulled this morning from Reuters, Palin says Obama friendly with terrorists. Notice how you can optionally maintain quotes and have them play with the words, perfect for telling a story.

What are your words saying?

Using Wordle Presoak to help make compelling word stories


6
May 07

Getting radical with UCD

Mark Detweiler from SAP has an exciting article – Managing UCD within Agile Projects – in the May / June issue of interactions. Last year I used SCRUM to manage a small development team on a high-profile project and one of the lessons we learned was that agile development does not work well with disciplines that are not also agile. The development is managed and completed in a series of sprints (we worked three, 30 day sprints) and the application was twitching before any formal UCD work had completed. This challenge applies to more than just User Centered Design (UCD); accelerating development is only so valuable. Detweiler’s UCD centric view offers a series of tips on making a better go at it, but the truth is this problem needs a more radical conception.

Detweiler offers three iterative UCD phases, Understanding Users, Defining Interaction and Designing the user interface. One observation is that UCD teams need to focus their energy on work that influences the current development sprint. Every profession has more they want to do. Delivering it apart form the product development positions their brilliance it to be ignored.

Understanding users is important, but it is not going to hold up development, so what is the minimal amount of work that needs to occur in order to have the development team be more considerate of the user context. Since we are iterating, maybe each sprint has a refinement of the user definition. One thing for sure, the developers will create a user persona if one is not provided.

Defining user interaction has to happen in a more collaborative manner with the development team. It is often just as easy to code the application, as it is to write the use case documentation. This does not mean use cases are a waste of time, but we need to ask what about the document is going to influence development. The time may be better spent with the UCD professional in the initial task breakdown meetings to help define interactions in detail. Having access to a UCD professional as the development occurs helps accelerate decision-making, but the goal is not to have the solution be perfect – it is to have a thoughtful solution, where lessons learned inform the next sprint’s interaction design. The UCD professional needs to own their role representing their stakeholders, but not constantly checking in to see if they are meeting expectations – there is simply no time for checking in and reporting back at every turn.

Designing the user interface as part of an agile development project is difficult. Short of starting this work prior to development, we need to find the opportunity in having the user interface be quick and dirty. An existing design system can address the developer’s need to assemble the application without having to worry about the overall application style. Assuming that a custom user interface is required, the designers need to be in the boat of iterative design. Save the Mona Lisa for a few iterations from now. If you want to influence this sprint’s user interface, worry less about the pixels and more about the styles. Design elements that telegraph where the design will be going. Development is iterative – it is assumed things will change so influence at each point, but save the masterful work for a time when it is not needed yesterday. Great work often takes time; why pretend it can be done in less?

All of this feels a little sloppy and this is why I think what we need is to revisit UCD and maybe the larger thought of user experience and how it is most affective in an agile development environment. This issue is exacerbated in innovation teams where often prototypes catch like wildfire and suddenly a solution with little to no formal UCD thought is at Version 1.0, causing pause to the value and role of UCD. We need to do more than apply more resource or slow down development to accommodate external dependencies. We need to do more than just the same thing in a slightly modified way. To be successful, we need to reinvent UCD or stop being agile.