For lunch today, Niki and I went with a couple of her friends from school to Tom’s Seafood Restaurant on the east side of Cleveland. It was my first real experience with dim sum, which are Chinese dumplings served either steamed or fried. It was all-you-can-eat, served buffet style — but instead of going up to a traditional American buffet (you know, with mysteriously wet plates and a sneeze guard), a group of servers wheeled carts filled with food to the tables.

I was really impressed with how different everything tasted. There were some things that were a little bit off the beaten path (even for me, and I really enjoy sashimi. For example, things like “chicken feet braised”, which, despite our best efforts, we couldn’t figure out how to eat. There were dumplings made with all kinds of meats, beans, and soy. They even had dim sum desserts, which were sort of like donut holes filled with custard. My favorite was probably the har gao, which are shrimp dumplings wrapped in rice flour.

The hardest part was trying to explain to the servers what type of food we wanted, because they didn’t speak too much English. (Although, I’m sure their English was better than my Chinese.) Niki’s friend Yoshie is Japanese, so they tried to speak Chinese to her but with no more success than any of the rest of us. :)

In the end, we all ate way too much. If you have the opportunity to try dim sum, expand your horizons!

One of the projects I’m currently working on is a WinForms application which interacts with some ActiveX controls via COM interop. One of the requirements of the WinForms app is that it be able to export the current view from the ActiveX controls to Microsoft Excel. So, no problem… each of the ActiveX controls has a method on it which copies the current view to the clipboard, and then via the Microsoft Office Interop libraries, I paste the information into a spreadsheet. Except one of the ActiveX controls doesn’t have the copy-to-clipboard functionality. Fortunately, it does have a function that creates an EMF image file on disk. No problem, I figured… I’ll just write a method in my .NET application that uses the ActiveX control to create a temporary EMF file, then loads it from disk and copies it to the clipboard.

I scribbled in the 10 lines or so of code required to execute this workaround, fired up the debugger, and clicked the export button. Instead of a shiny Excel worksheet, I was presented with an exception:

Current thread must be set to single thread apartment (STA) mode before OLE calls can be made. Ensure that your Main function has STAThreadAttribute marked on it.

Crap. So what’s going on here? The secret is that the Clipboard class in .NET still requires COM interop to execute the OLE call necessary to move information to the Windows clipboard. I didn’t see this problem when calling the copy-to-clipboard functions on the ActiveX controls, since they were written in unmanaged code and didn’t need to cross any managed/unmanaged boundary to deal with OLE. What confused me is that my entire application is designed to work with COM interop anyway, so I’d already set it to run in single thread apartment mode. What’s the problem?

After some digging, I found an article on C# Corner that contained a line that I really didn’t want to see:

…all ThreadPool threads are in the multithreaded apartment…

Double crap. In accordance with good interface design strategies, the Excel export feature is done in the background using a ThreadPool worker thread. Using the ThreadPool is a great low-effort way of spawning background processes, but I’d designed myself into a corner on this one. You can’t (or at least Thou Shalt Not) change the information on a ThreadPool thread, since technically you don’t own them, and they can be reused. And, since some exports take several seconds to run, I couldn’t execute the Excel export code in the interface thread without hanging it for a long period of time.

So, in the end, I worked around it by using Control.Invoke() to call the function that does the actual copy-to-clipboard work from the background worker thread Usually, this call is used to avoid violating the One Rule (Thou Shalt Not update an interface via any thread other than the thread that created it). However, this use allows me to interop with the ActiveX component from inside the STA thread — while still running the bulk of the export code in the background. This lets me keep the interface responsive and lets the cancel button work, but let me escape re-writing my thread model.

Whenever the topic of interface development comes up, I’m always surprised to see most software engineers cringe as if they’re being told they need a root canal. Almost all modern applications require some sort of graphical user interface, and yet the UI is commonly the last consideration of development. Worse yet (particularly when it comes to web development) the user interface is often created by a graphic designer who isn’t familiar with software development. The resulting separation that occurs between the application’s internals and its interface can cause serious problems with the project.

The Black Magic of Interface Development
This aversion by software engineers to interface development is unfortunate, and can endanger the success of a software project. Since the UI is the only area of the application that the user interacts with, to them, there generally is no difference between the concept of the interface and the concept of the application. Internally, the software may be well thought out, extensively commented, using the most cutting-edge technologies and practices available — but if you make Aunt Marge enter commands at a text prompt, she’s not going to care. She’s going to think your application sucks. And she might hit you with her cane.

Many software developers look at interface design as a type of voodoo, except that all that sticking pins into your monitor is going to accomplish is leaking liquid crystal. Some of this is undoubtedly a result of the lack of formal training on event-based programming in software engineering curricula. During my entire time in college, only a single class contained any projects requiring a real event-driven graphical user interface. Then again, my education was based in C++, and I appreciate the powers-that-be not subjecting the students to the bone-crushing, mind-warping horrors of MFC. Maybe this is one of the (very few) benefits of moving university-level curricula to Java. :)

Simple Structured Interfaces
The secret? Interfaces aren’t nearly as complex or foreign as many developers make them out to be. In my experience with them, I’ve gradually converged on some general guidelines which I’ll share with you in this article. Interfaces can be easily separated into a small amount of “states”, which represent the different situations that the interface can be in. Different conditions in the application can cause the interface to move between states. This idea is represented in computer science as a finite state machine. For those readers with a strong theoretical background or a masochistic personality, I suggest reading the Wikipedia article for more information. A basic understanding of multi-threading is also assumed from here on, but don’t worry, I’ll be gentle.

If reading about complex abstract theories isn’t your idea of fun, don’t worry. (This just means you’re a software engineer, not a computer scientist. This is not a bad thing.) It all starts with the state diagram below, which illustrates the basic structure of an event-driven user interface. (Click the diagram for a larger version.) If you’re not used to state diagrams, or even the idea of application states, don’t be intimidated; I’ll walk through each step in more detail. At the end of the article, you’ll be able to impress your friends by saying the phrase “finite state machine” at parties. (Note: any swirlies you receive as a result are not the responsibility of the author.)

The circles in the diagram indicate the various states of the application’s interface. The arrows connecting them indicate ways to move between states. Roughly, entering a state means that a specific method inside the application will be called. Arrows in the diagram indicate pathways to move between events. Arrows with labels indicate an event must occur in the application for the state to change, whereas arrows without labels indicate that the state change will occur naturally once the work in the current state finishes. Often, the labeled arrows represent a collection of similar events that can occur to move the interface between states — for example, a collection of button click event handlers.

In an event-driven programming model, the firing of events essentially result in a certain method or collection of methods being called. In a .NET application, this method will match the signature of a delegate, which will be defined as an event handler. (For a solid introduction to .NET delegates and events, see here.) For the web developers among us, events are generally handled by a JavaScript function, which are designated as event handlers through HTML attributes like onclick.
The Start State
As you may have guessed, the application begins in the Start state. As it starts up, it creates the components of the user interface during the construction step, which moves the interface to the Initialize state. Whatever preliminary work must be done to make the interface display its default view is done in this state — for example, the application must load information into list boxes, and apply any user preferences that may exist. In “thick” applications (rich interfaces), this is often done inside a constructor. In “thin” applications (web interfaces), this is done during the HTML rendering step when the page is loaded.

The Idle State and the Lazy Ninja Interface
Once the interface has completed initialization, it enters the Idle state. This is where it will spend the majority of its time, simply waiting to respond to incoming events. The loop in the diagram pointing directly back to the Idle state indicates that the application will remain in the state by default, and only move to another state when an event occurs. In a rich environment (either a thick client, or an AJAX-enabled thin client), the application has the ability to process external information while the interface is considered idle. This responsiveness results from the power of multi-threading, but a deeper discussion of concurrency is beyond the scope of this article. Incidentally, the lack of this additional thread is why “old-style” web applications without the magic of AJAX generally require user interaction (say, a page refresh, or an HTML form submission) to update the interface.

An important guideline when developing interfaces is that you want to keep the interface in the Idle state as much as possible. The more time the thread controlling the interface is idling, the more responsive your interface will be, and the more rich of an experience will be provided to the user. If you’re into anthropomorphizing inanimate objects (for example, you’ve been known to talk to plants), you can think of the interface as being lazy. It doesn’t want to do any work itself. Instead, it just wants to know when it needs to pass the work on to another segment of the application. When a request is received, the interface has to immediately get motivated with ninja-like speed and dispatch it to someone who can handle it. Otherwise, your job as a developer is to give your lazy ninja interface a bag of pork rinds and a nice comfy spot on the couch.

The Feedback State
There are three events that will move the interface out of the Idle state. The first possibility is that the user will interact with the interface, such as pressing a key or moving or clicking their mouse. This moves the interface into the Feedback state. In this state, the interface provides the user with graphical and/or audible feedback to indicate that it has understood what the user is trying to tell it. For example, the interface may change the color of a button and emit a “click” noise when a user clicks a button. From a psychological perspective, providing constant positive feedback helps to keep a user engaged in the application, and improves their overall experience significantly.

The Response State
After the user has been presented with feedback to indicate that the interface has received the user’s command, it immediately moves into the Response state. The task during this state is to fulfill whatever request the user has made of the software. In a rich environment, this generally means dispatching the request to a “worker” thread, which will do the necessary processing in the background. This means that the thread controlling the user interface won’t be tied up doing processing, and can remain responsive to provide the user with useful gizmos like progress bars and cancel buttons.

The Synchronize State
Once the user’s request has been fulfilled (or at least dispatched to a worker thread), the interface needs to re-assess the situation. What has changed in the environment, and how should it affect the tools presented to the user? This is the state where buttons are be disabled (”grayed out”), status messages are updated, data that has changed is re-loaded into the interface, and anything else that has to be accomplished in order to bring the interface into sync with the status of the application is done.

Another way the interface can enter the Synchronize state is if it detects an external event. Usually, these will be fired from a background thread that is monitoring some external conditions, and when these conditions change, an event will fire indicating that the interface should be updated to reflect the updated information. For example, in a chat application, there is a thread in the background that listens for the person on the other end to transmit a message. When that message is received, the background thread fires an event that tells the interface “Quit sitting around! Draw this new text on the screen!” The interface will jump directly into the Synchronize state and update itself with whatever new information is available, so the user is immediately aware that pR1Nc3ssSuzi3 has just sent them a message saying “omg lol kthx bye”.

Once the interface has synchronized itself with the current application state, it immediately grabs its bag of pork rinds and re-enters the Idle state, where it will wait for the next event it has to respond to.

The Cleanup State
The final way that the interface can leave the Idle state is when it is “disposed”. This occurs when the application is shutting down, or the interface is no longer needed (for example, when a chat window is closed). In this state, the interface should do whatever necessary to clean itself up and prepare for destruction. This is where the interface should “un-subscribe” from any events that it is listening to if the language you’re using requires it. In this state, you should also make sure that no work is being done in the background, and if so, you should either wait for it to finish or let the user decide to cancel it.

The Stop State
This state is reached when the components that compose the interface are destroyed in memory. Obviously, since the interface no longer exists, it doesn’t do anything in this state. Unless it comes back to life as a zombie interface and starts eating RAM like real-life zombies (?) eat brains, but that’s a different topic for a different day. (Actually, your interface can become “zombified” in .NET if you interop with native COM components without cleaning them up correctly. Be sure to free them while you’re in the Cleanup state.)

Summary
In this article, we’ve talked about voodoo, lazy ninjas eating pork rinds, real-life zombies, swirlies, and also user interfaces. A few things to keep in mind:

• Keep your interfaces lazy, but give them ninja-like reflexes. Accomplish this by not executing long-running processes in the interface thread.
• Each state in the interface won’t be represented by a single function in your code. In particular, in strong-typed languages like .NET ones, you’ll need more than one event handler.
• However, try to make your code cohesive (and not repetitive) by keeping business logic out of the event handlers themselves. Create a small number of methods that will do the work required, and call these methods from your event handlers. This way, if you add a button that does the same thing as a menu item, for example, both event handlers can call the same method without repeating the actual logic twice.
• Provide your users with constant and consistent feedback to keep them engaged in your application. No, this does not mean you are allowed to use the <marquee> tag. Progress bars and cancel buttons are great, though.
• Interfaces are not black magic. You can create good ones without being a voodoo practitioner. All it takes is some patience and a little bit of experience.

A friend sent me a link to this article describing an amazing use of Google Earth, to document the Israel-Lebanon conflict with pinpoint accuracy. The creator of the KML file that adds the information to Google Earth collected the information from various news sources and combined them to give an incredible perspective on the violence in the Middle East.

This application of technology to collect and disseminate news during an ongoing conflict has never reached this level before. Throughout history, battles have been recorded and chronicled during wartime, but not in real-time and not with this level of geographical precision. I’m curious of the impact this might have to counteract propaganda and disinformation spread by opposing forces during a war; with the possibility of organizing information in such a geographically-centric way, tech-savvy citizens in the warzone could have a better understanding of troop movements and behavior, making rumor-mongering and panic less likely. Obviously, this could still be tainted by spreading disinformation to the news sources whose news is subsequently aggregated, but it would definitely be more difficult when comparisons were so readily available.

This is another example of the new distribution methodology present on the Internet today: instead of a small amount of large organizations creating content, large amounts of small organizations (or individuals) generate the information, which is then aggregated and sorted in a way such that the most relevant material bubbles to the top. This is the same methodology used by Google News, digg, del.icio.us, Technorati, etc. It will be interesting to see where it is applied as it continues to spread further into the mainstream information sources.

Yahoo! announced the other day that they would offer MP3s unencumbered by DRM (digital rights management) technology.

Everyone together now: WHAT TOOK SO #@!$@ LONG?

Sure, they’re only offering one measly song, and it’s Jessica Simpson, no less. Not exactly my style of music, but hey, it’s a start. It’s been said many times that the entertainment industry will ultimately lose the war over digital copyright law because they “can’t compete with free”. The assertion is that if they can’t retain control of their intellectual property, nothing else matters.

What a load of crap.

Even today, years after its original inception, MP3 is one of the most powerful technologies on the Internet. Wrapping an MP3 in DRM is a lot like making a Ferrari capable of going 500 miles per hour and getting 100 miles per gallon, and then fitting it with square wheels made of concrete. Oh, and you can only drive it down certain roads. And on some models of the car, you have to keep paying a monthly fee, or the engine won’t start. And, if you get caught letting anyone else drive it, you could be fined up to $250,000. Oh, yeah, and if you get tired of the car, and you want to make a switch, don’t even try to sell it. It doesn’t really belong to you, see… Ferrari still owns it, even though you paid for it.

Ridiculous, right? Why doesn’t the recording industry see that even without the terrible press they’ve been getting for the past few years, they can’t possibly hope to compete with illegal MP3 downloads. How do they expect to compete with a product that is both more valuable (since it doesn’t have DRM restrictions) and costs less?

The answer: lawsuits. Lots, and lots, and lots of lawsuits. I’ll tell you what, RIAA. Let’s make a deal. You give me half the money you have earmarked for legal action against music fans, and I’ll come up with a solution for you. Better yet, give it to the artists that work for you, or use it to promote their music. Even better, use it to find bands that have real talent, that aren’t just marketable Ken and Barbie dolls, and turn us into their fans.

After all, isn’t that your job — promotion? Or is it just to bite the hand that feeds?

Everyone has a price. Sometimes, though, the cost isn’t measured in dollars and cents. I recently read the article Top Ten Ways to Motivate Geeks, and being a geek myself, I got thinking about the inverse argument: what can a manager do to piss off their technical underlings such that they are less productive? After talking with a few of my fellow geeks, I present, in no particular order, our humble list of ways to de-motivate the technophiles that work for you. Note that I use the word “geek” to represent anyone that works with technology on a day-to-day basis, but particularly people working in software development or IT.

Disclaimer: This is not directed at my current employer or its management, and is not intended to describe any single person or any single company. It was compiled from the opinions of several geeks working at several companies in multiple states. Please don’t fire me. :)

1. Insult their ability or intelligence
The best geeks are confident, almost to a fault (sometimes not almost). As it is with any creative person, they don’t do what they do just to pay the bills. They don’t just do what they do, they are what they do. If you directly insult their programming ability, you’re not just insulting their code, you’re insulting them as a person, and as an artist.

Sound too wishy-washy? Maybe so. The best have a thick skin, and can absorb criticism pretty readily. I’m not suggesting that you should avoid criticizing them entirely — knowledge is a combination of instruction, experience, and feedback. Without a critical review once in awhile, they’ll just start assuming that they’re the best of the best. (This is undoubtedly why peer code reviews are becoming popular in XP environments.) Instead, restrict your criticisms to the product of their ability, not their ability itself.

Another thing to remember: misusing terminology implicitly insults a geek’s intelligence. “Oh, that problem can be solved easily! Just re-instance the variable with a generic!” Huh? If you don’t know the answer to a problem, don’t offer your opinion — particularly if you’re not a geek yourself, or if you don’t have the domain knowledge required to contribute. Don’t believe that by clobbering together a bunch of language keywords that you’ll convince anyone (particularly someone who understands what all those words mean) that you know what you’re talking about.

2. Take credit for their work
This is directly related to #1. Remember, a good geek’s work is a labor of love. When you take credit for their work, you might as well make a pass at their significant other while you’re at it. If you’re going to take credit, limit it to credit for directing them only, and leave any technical work that you weren’t directly involved in out of it. In fact, for bonus points, mention the geeks’ contributions to your superiors. Good geeks are team players, but they’re also proud, and it bothers them to never get recognition except as “a member of (manager)’s team”.

This particularly holds true for creative decisions. If you allow your geeks creative latitude (and you should, see #5), reward them when they take advantage of it and come up with a great idea. They will remember it, and they’ll be more apt to come up with creative solutions the next time around. Likewise, if you take credit, they’ll remember it — but they’ll start asking themselves why they bother to come up with good ideas anyway.

3. Don’t communicate enough with them
Geeks are information junkies. This is what lets them constantly improve their skills and continue on the search for better ways to solve problems. This also means that they want to know what’s going on in the company around them, and particularly the projects they’re involved in themselves. Within reason, answer their questions to the best of your knowledge. This doesn’t mean you should divulge things that the geeks shouldn’t know (for example, it’s probably best you don’t get into financial specifics), but they should be made to feel like they’re “in the loop”.

The best geeks aren’t just geeks, they’re also strategists, and capable of understanding business. They want to know what the company’s primary goals are, so they can adequately improve their skills to follow suit, and align themselves with the right projects. This is more about self-preservation than anything else, but it also stems from a natural desire to be on the cutting edge of everything. The grand majority of geeks are very forward-thinking, and aren’t interested in using technology that’s stagnated. They’d much rather try out that new circular saw than be comfortable working with the same handsaw they’ve been using for years.

Although you should be sure to communicate with them, you should never, ever, ever lie to them. Geeks spend the grand majority of their time dealing with factual data that is constrained by logic. They are also likely to be able to follow logic through multiple complex stages. If you do decide to lie to a geek, don’t ever go back on your statements, and don’t try to layer more lies on top. Geeks will be able to reason their way through your nonsense, and being naturally cynical, they will start to disbelieve everything you say.

4. Communicate with them too much
While it’s true that geeks want to be in-the-know, they don’t want to know about certain things (even if they sometimes think they do). For example, unless they’re wearing a sales-engineer hat, they should be left pretty well out of the sales cycle. Their deterministic nature will make it more difficult for them to understand why a client doesn’t see that their software is fantastic, because the code is so clean and it uses this really cool idea for caching that they came up with in the shower one morning.

Likewise, the job of a salesperson is to promise everything under the sun to the client, and then produce just enough of what they promised so the client doesn’t get pissed off. Don’t tell the geeks what the sales staff is promising customers, or you’ll start to see bottles of Maalox show up on their desks.

5. Stifle them
Software development is a lot like art. When confronted with the task of drawing a likeness of a person, anyone can draw a stick figure. Likewise, anyone can learn VBA in a few weeks and jam together something to track their coffee expenses. This artistry is one of the reasons why software engineers and mathematicians are not interchangeable. Geeks might not readily compare themselves to artists, but they’re much more on that side of the spectrum than, say, statisticians.

The worst possible thing that you can do to a geek is say: “Here’s a fun and interesting problem, but because of (insert illogical reasons here), you have to solve it in this way only.” Even worse, keep them in the dark about the restrictions until after they’ve finished a creative and effective solution, and then give them some nonsensical reason why it can’t be used, like “Our customers know Java! We can’t possibly sell something that’s not written in Java!”

6. Give them impossible deadlines or ask them to work ridiculous hours
Nothing destroys morale (and by extension productivity) faster than hopelessness. Everyone wants to feel like their work is going to make a difference, so if your geeks believe that success is impossible no matter what they do, they’re not going to bother. Bad morale is viral, too, so it only takes one person on a team to become discouraged for the entire team to be in danger. Worse yet, once they’ve caught the hopelessness bug, ask your geeks to work 14 hour days to meet the impossible deadline. Likewise, if you don’t set reasonable deadlines and instead ask that everything be completed yesterday, don’t be surprised when your geek ignores new requests and continues to work on what they’re already working on. Remember, you said that task was absolutely vital two weeks ago. If every task ends up at #1 priority, it falls to the geek to prioritize them, and that gives them the opportunity to prioritize based on what they want to do on a given day rather than what’s best for the project.

That being said, crunch-time is a reality in almost all software development projects. It’s impossible to perfectly estimate the amount of time required. Dozens of books have been written on the topic and yet no one really has the magical formula. Sometimes the timeframe will be have to be compressed in order to get the contract, or the customer will change their requirements, or there’ll be a flaw in the design that makes you take a couple steps back, or whatever. Asking geeks to work lots and lots of overtime will not yield more work. In fact, there are some very good arguments that stretching the schedule past 40 hours actually results in lowered production, because fatigue creates “negative” work which has to be corrected later during more lucid periods.

7. Make them work on “soul-crushing” projects
Every geek has been involved in, or knows someone who has been involved in, a project that can only be described as “soul-crushing”. The first requirement for a project to be soul-crushing is that the people that work on it consider it to be boring. Geeks, being creative individuals, want to be intellectually stimulated by their work. If they wanted every day at work to be the same, they would get a job driving a truck, or digging ditches. (This is of course not meant to denigrate the truck drivers or ditch diggers of the world. It’s just not my thing.)

There is a difference, however, between projects that are simply routinely boring and projects that are soul-crushing. To rise to the next level of pain, a project must meet other criteria. Here are some examples:

• It requires the creation of hundreds of similar components that are just different enough that they can’t be adequately abstracted.
• Just the development will stretch on for years, with no end in sight. Then, the geeks get to support it for the rest of their natural lives.
• It’s guaranteed to lose the company money. Everyone knows, and no one cares.
• It’s built on a terrible foundation, or it tries to re-use or re-purpose code that should never have seen the light of day in the first place.

Projects don’t start out being soul-crushing. Some projects are horribly uninteresting, but any project can be exciting to the right people under the right conditions. The existence of soul-crushing projects in a company can be a sign that the company is in trouble.

8. Make them misrepresent their work
Salespeople are capable of walking into a company and telling boldfaced lies about the stability or the features of a product. That’s good, because that’s their job. It’s a little like dating… you lie to the other person until they love you for who you really are. Without salespeople, the money would run out before the product *is* stable enough to sell, and the geeks couldn’t cover the monthly bills on their MMORPG habit.

There’s a reason geeks aren’t good at sales. The best geeks are hopeless perfectionists that can never be fully convinced that “good enough” is good enough. When they look at their work, they see it “end-to-end”, through the sexy interface to the flaky API, through to the buggy back-end and the twisted database with 14 bolt-ons that one day, they really, really want to fix if they could just have a little more time before it ships.

This understanding of their software (which you might call “x-ray vision”) allows them to do their job, but at the same time, it makes them doubt the product. They know that there’s a kludge in that function around line 43. They know that the memory requirements are such that the software won’t scale past about 10 concurrent connections given the current hardware. Worst of all, they know that the software hasn’t been tested yet, or if it has been tested, it has dozens of outstanding issues in the bug tracker. None of these things are apparent when all you can see is the sexy interface, which is as far as the salespeople and the customers see.

Unless sales is part of their job description, don’t make geeks lie about their work, not to the customer, not even to their co-workers. Even worse, when they tell you the software’s not ready or that guy in marketing that promised that customer a holographic smell-o-vision interface is nuts, don’t…

9. Ignore them when they know what they’re talking about
Sometimes, there’s a fine line between confidence and egotism. Sometimes, there’s a fine line between giving an honest opinion and crying wolf. If a geek raises the red flag and starts waiving his/her arms at you, pay attention to them at least long enough for them to try to plead their case. If you don’t believe them, don’t just ignore them — reason with them why you disagree. Remember that geeks are, above all else, built on logic. If you think they’re being overly emotional, chances are there’s a reason for it, and you might want to pay it heed. If you don’t take them seriously, it raises the chance significantly that they won’t take you seriously next time.

10. Don’t adequately protect them from customers
First, it’s important that you remember that the customer is not always right. Sometimes customers make stupid requests because they don’t understand what they want, or they don’t understand what they’re asking for. Some customers think using the telephone, and ask for something that they feel will solve a problem before reasoning out the situation on their end. It’s important that you don’t make geeks field too many of these requests. Unless everyone is sitting around playing World of Warcraft because there’s no work to be done, one of your jobs as manager is to triage the resources that you have available.

Unless they’re paid to do so, geeks don’t want to have to figure out what clients want for themselves. They want you to translate the “client-speak” into objectives, which they will then translate into code. If the geeks are spending their time on the front lines, they’re not spending their time in the trenches, making the product better. If their business card doesn’t say “support tech”, don’t put them on tier 1, because this will only result in less code being generated, and one very pissed off developer.

The other day I read an article written by Mike Vanier, a faculty member in Caltech’s computer science department. In it, he discusses what he calls “language scalability”, or the ability of a programming language to support large-scale development.

I have to agree with almost all of the points in the article, particularly his discussion of garbage collection. Having written large applications in both C++ and C#, I can attest that memory management is by far the most dangerous part of software development — not to mention the most tedious. It’s not too bad when you have a deterministic understanding of where the memory is being used, but particularly when you get into object-oriented development in C++, and you generate large object maps or trees in memory, the likelihood that you’re going to correctly free memory approaches zero.

Another thing that I find extremely necessary for me to consider a language “scalable” is the availability of easy to use flexible data structures, such as linked-lists and hashtables. When writing data-driven applications, most of the time data will relate in a non-deterministic way — that is, elements will often relate 1-to-N where N is unknown. In these situations, it’s vital that the language you’re using not force you to define the amount of memory you need before you need it.

Take, for example, the task of loading data out of a database into some type of structures. If a language requires that you tell it how much space you’re going to need before you load the information out of the database, one of three things will happen:

• You’ll allocate a bunch of memory up to the maximum that you “think” you’ll need. Of course, if you’re wrong, you’ll end up with buffer-overflow errors, and if your language sucks, you won’t have any sort of bounds checking on your arrays. Not only will your program get crashy, it’ll open up some nice security flaws as well.
• You’ll come up with a brilliant way to figure out how many elements you have, and allocate precisely the right amount. Unfortunately, this will almost definitely require an additional call to the database (for example, a COUNT(*) query), which adds overhead. Also, I hope you’re not going to let your user add or delete any elements, or that fixed array is going to get reallocated pretty quickly.
• You’ll start to write your 143rd implementation of a dynamically-resizing array-based linked list, chew on your keyboard a few times, and then give up and take a job as an alpaca farmer.

At any rate, while I do agree with the grand majority of what Mike is saying, I think his argument holds up better in theory than in practice. In the commercial sector, large, complex software projects are generally written by a large number of people. As a result, I believe that for a language to truly be scalable in the real world, there must be a “critical mass” of developers that understand the software — so many that enough of them can be assimilated into the same development team. When a company makes a choice to implement a large project in a certain language, they’re making a long-term investment in the life of that language.

A lot of the languages that Mike refers to as scalable, such as ocaml, lack the critical mass necessary necessary to make it worthwhile for companies to make an investment in using them. Even languages that are closer to mainstream, like Common LISP and Eiffel, don’t have enough developers in support of them to make them useful in a commercial environment. This isn’t to say, of course, that you have to write every commercial application in Visual Basic just because there are more VB developers around than anyone else. (If this was the case, I would already be an alpaca farmer.) However, it’s much more likely that a company would take the plunge and support a language like Java or C#, which are arguably “less scalable” than a fringe language like ocaml, simply because the talent required to develop in them was more readily available.

Also, for the record, I can prove that LISP doesn’t scale well at all: the parentheses keys would snap off your keyboard after about 50,000 lines of LISP. :)