How’s your agile process working for you?

Are you practicing an agile process in your team? Do you still find your team working through the night? Do you still find your software to be rigid, fragile and immobile? Do you still find changes of requirements to be nightmares to the developers? How do you suppose those situations are possible with an Agile process?

Unfortunately, this is the situation in almost all Agile practicing and Agile preaching companies we have around in Sri Lanka. I’ve crossed my paths with quite a few developers from these companies and they all have the same story to tell; agile for them, is the same old cycle, without the documentation and without a designing phase - which makes their development life worse. Non-technical project managers are still around, being called by other names, and to add to the misery, now the customers bring the pressure directly to the developer - because it’s “Agile”.

Is Agile a bad idea? What’s going wrong? Why can’t we make it work? Have you seen the list of names in the Agile manifesto at http://agilemanifesto.org/? Do you think those are the kind of people who would invest on a failing process? Besides, they’ve been practicing and teaching it for over a decade now. So if there’s anything wrong with Agile, it’s more likely in the way we practice it.

Management elements such as deadlines, budgets and sucking up to customers’ aside, we developers too need to change some fundamental ideologies towards development in order to make Agile, well Agile. It is not to say that the operational model doesn’t matter, indeed, that’s what makes Agile possible. Someone should be available to the team to represent the customer, acceptance tests should be automated and made available to the developers when feature requests / changes are made; team should be responding to changes than follow a plan; team should be able focus on getting a working product out instead of documenting etc. etc. which all agile gurus will tell you; but the part they don’t speak about; the part the folks came up with agile manifesto took for granted; the part that is responsible for more than half the agile failures in Sri Lanka is with the developers.

One misconception most developers in Sri Lanka have about Agile development is, it is coding without design or documentation. The other misconception is to think getting software to “work” means getting the functionality out one way or another. The Agile philosophy has very different interpretations for these and is defined with the implied expectation of strong object oriented concepts and design principles of the developers.

By the term “a working product” what’s meant is a functional, testable and maintainable product. Not just functional. If a piece of code provides the functionality required, but only the developer himself could read and understand how it does that, it is not a working product by Agile definitions. Developers are to get the code to work and then refactor till it’s readable, not duplicating code and as simple as it could be but not simpler.
Refactoring to make code readable, non-duplicative, simple and free of unused design complexities and code is a constant practice that developers are expected to follow whenever they code something.

The design focus of Agile is much more implied than explicit. In each iteration, Agile focus on getting a working product to the customer. The team narrows their focus only on the features that are planned for that iteration making it as simple and expressive as possible; yet not losing the overall picture - or at least what exist of it at the time. This practice makes sure that the design never gets complicated than necessary and serves to the problem at hand than expected (but volatile) problems of the future. One of the problems in designing the whole application at once is that the architect is force to foresee requirements of the future; with requirements ever changing and little understanding of them to begin with; this is not a task someone could get 100% right. That’s why Agile promotes this simplistic approach which could accommodate changes more easily.

Unit Tests for agile developers are a design practice than a verification step. Writing the non-compiling unit test is defining the contract for a component. Developer should do due diligence and consider who are the consumers and think how the consumers would want to use a class. Note that the tests are written for the public interface of the class, the tests care only about the functionality exposed through that. Once the developer get the unit tests to pass, he can then go on refactoring his code to be simple, readable and non-duplicative - one step at a time - without worrying about breaking the functionality unknowingly; because the tests are there to guard against that.

The next aspect of design developers ignore in Agile is how exactly the changes are accommodated. Remember that agile designs are as simple as possible meaning that when code is first written, it is not accommodating flexibility for “potential” change. But when the first change comes, the developers should follow the object oriented design principles and separate the changing elements of code from the non-changing elements of code, evolving the design in that part, so that a second change in the same area would be easier and simpler than the first. This practice prevents the code becoming stale over repeated changes, still keeping the code as simple as possible.

As always, developers should keep themselves up to date not only with the new technologies but of the time proven concepts; and the organizations who wish to harness the benefits of any methodology should invest on getting their developers in par with that methodology.


References
Agile Principles, Patterns, and Practices in C# By Robert C. Martin, Micah Martin
Agile Software Development: The Cooperative Game, Second Edition By Alistair Cockburn
Test-Driven Development By Example By Kent Beck

Kubuntu 10.04 LTS

I've used Linux some years ago. In the days of Red Hat 6.4 and then 7.1. My career in .NET development and probably some frustrations with compatibility issues with my hardware at that time, made me switch back to Windows and never look back. I've since used and promoted Windows and had perceived all Linux compilations to be at the same level as Red Hat 6.4.

I did downloaded Fedora 11 in 2008 or so due to frustrations over Windows Vista but somehow never installed it. When Windows 7 came along, I thought Windows days are here to stay. But Janaka at office kept talking about Ubuntu once in a while and that made me want to give it a shot.

I received my CD of Kubuntu 10.04 LTS from good people at Canonical couple of days ago. I installed it on a VM on the same day and was impressed with the user experience. Today, I installed it on my father's laptop and boy! I'm loving it.

So, those of you who are using pirated copies of windows making the excuse that Linux is not in par... Take another look at Kubuntu 10.04. It's way better looking and faster than Windows 7 and is guilt free!

The DigiKam application is a feature rich image editing tool at par with Nikon's Capture NX. Open Office has come a long way to be a good office suite and Google Chrome lives by the name of Chromium in the Linux world. I haven't looked at any of the IDEs yet, but I'm sure there are some that could compete with Microsoft's Visual Studio.

So, this is me, a pro-microsoftee asking you to try out Kubuntu. Believe me, it's really nice.

Ironic

This is funny; but unfortunately, I think I signed somewhere agreeing not to discuss these things in detail in public. So this is what I guess would be okay to write here.

I'm writing code for the next version of an application server that most of the .NET developers would soon start to use. And, that server is going to have a new feature integrated in to it by default - out of the box. I'm right now trying to get that done - get that feature integrated and made default through its post setup configuration program.

Since these two products were in the existence for sometime I thought someone must have done it before - not out of the box, but through some post setup customization. So I Googled... And the first hits I get are news items that say "The X server would have Y service by default - out of the box in its next release"... :)

365 Photo Project

I've been taking pictures for few years now; but never had the will to do a photo per day project. But after buying the S90, I thought I would do one. A whole 365 day thing!!! Let's see if I could keep that kind of a commitment.

I've counted two days so far.. publishing at http://365.photoweaver.net. I'm struggling to keep up already... But let's see; If things go really bad, I might post photos from my phone :)

XML Validation error: Data at the root level is invalid. - could be a UTF BOM issue

UTF has an optional Byte Order Marker at the beginning of the file. The UTF encoding classes included in the .NET Frameworks previous to 4.0 by default did not write this BOM. Hence, most applications we wrote to read UTF files does not handle the BOM. :)


I recently wrote a tool using Visual Studio 2010  to generate a UTF8 encoded XML file to be consumed by another application which read the XML file using PowerShell. That app failed with an error "XML Validation error: Data at the root level is invalid. Line 1, position 1." Of cause, opening the file in Notepad2 didn't show any suspicious characters. Some educated colleagues pointed us at the possibility of BOM; which proved to be the case.


The fix is relatively easy. The UTF encoding class constructors takes in a boolean parameter encoderShouldEmitUTF8Identifier. Just pass false in to that.




        //
        // Summary:
        //     Initializes a new instance of the System.Text.UTF8Encoding class. A parameter
        //     specifies whether to provide a Unicode byte order mark.
        //
        // Parameters:
        //   encoderShouldEmitUTF8Identifier:
        //     true to specify that a Unicode byte order mark is provided; otherwise, false.
        public UTF8Encoding(bool encoderShouldEmitUTF8Identifier); 


Example using a XML writer:



XmlTextWriter writer = new XmlTextWriter(exportPath, new UTF8Encoding(false));

Why Google search is still the best

I use GhostDoc for code documentation.

Today, I was working on a different machine and found that it doesn't have ghost doc installed. I opened up a browser and typed ghostdoc in the search box and hit enter. For my surprise, I couldn't find the SubMain site on the first page of search results. Puzzled; I looked what's wrong to find out that the search engine was Bing...

Open up google and search GhostDoc and the first hit is the correct download site.

Search Google for "Mahasen" and the first hit is this blog.. it even suggest "Mahasen Bandara"; Bing doesn't fetch this seven year old blog till the 9th page; But it fetches my  two weeks old site photoweaver.net in the first page.

I don't see myself using Bing anytime sooner...

Life goes on

I've been working on a DSL (Domain-Specific Language) in Visual Studio 2010 RC to design deployments of MOSS Search Farms.

There were a lot of things that we solved, which we couldn't solve just by few minutes of Googling, basically because they've changed the architecture of DSL in visual studio 2010 but haven't documented/blogged them fully yet. Every time we crossed something of the sort, I thought I'd blog about it, :) But didn't.

We released a Beta of the tool last Friday to the internal tech community, and the feedback we've received so far is pretty good.

Hopefully, sometime this week, I'd start posting some posts about DSL in Visual Studio 2010 RC, and if/when we release the tool, I'd hopefully blog about that too..

The way we screw requirements up.

My previous job was at a small software firm. They had no software development process at all. There was couple of people who mainly interacted with the customer to get the requirements. They spend few days with the customer discussing; and then sit with the developer for half an hour and ask him to write it! No documentation; no diagrams; no nothing. Write this. Once in a while they’d check what’s going on; show the customer what’s going on and change few things here and there and that’s it!...

Needless to say; we never made correct estimates; we never delivered what we initially thought the system was and very little number of customers came back. The applications were not designed at all. They were ad-hoc code with only the blessings of the skills of the developer.

That sucked… So I joined this company which has a pretty matured software development process. It defines how to capture requirements in different scenarios and conducts periodic audits to make sure that everyone is on track. Pretty neat eh? This all works well when everyone involved in it is committed and competent (almost all committed people are competent and almost all un-committed people are incompetent. Once in a while there’s a committed but incompetent person and a competent but un-committed one.)

But requirement capturing was a nightmare in the last two projects I worked in. Why? Some smart-ass thought anyone with CIMA or a MBA, who’s fluent in English, could be a BA. They simply didn’t know reasoning or probability. They just wrote what they heard. (I’m pretty sure they didn’t understand a thing what the customer wanted. But they didn’t know how to ask a question so that there won’t be ambiguity)

So what’s the end product? A big pile of documents that has no value at all and several wasted months (this is no small deal; one project had two BAs who wasted three months and the other had 5(!) BAs who wasted 10 months!!).

So; to think of it; the first company did less damage to the customer and the developers than the second.

So what really works?
A requirement capturing process practiced by professional BAs or Software Architects. That works really well - I’ve seen that. So; if you have any stake in a software project, make sure that you are talking with someone who’s capable of translating your requirements in to something that developers understand. Otherwise you’d just be wasting time and money - a lot of it (those 5 BAs wasted at least about US$1 million).

Can an offshore software service company benefit from the current financial crisis in the world?

I believe yes. But there’s a certain formula.

First of all; the company should be well managed up till now. There should be 95%+ utilization level and they should already have 2~3 good clients. The engineering staff should be above average, comprised with people who are able to make a difference in a project. A well defined software development process must be being practiced. There should be one or two architects who could really stand by a white board and convince people. Company should be small - 40 to 100 head count.

Just because there’s a financial crisis, it doesn’t mean there aren’t any projects out there. Companies in the US are cutting jobs and eliminating all possible expenses. But they still have projects which are critical for their business. So offshoring is a very cost effective solution with lesser liability for them. If someone can offer them a cheaper, better and faster solution - they’d go for it.

A bigger offshore company is likely to have,

• Management overhead
  
• Under utilization 
  
• Smaller percentage of above average engineering staff.
  

Therefore they are unlikely to be able to offer a competitively better price or a service than a smaller company with good staff.

For a company who can offer a good service for a lesser price (the formula in above); It’s the right time to grow. Most companies are taking a hit and putting raises and promotions on hold. So it’s a great time to hire good people. You shouldn’t hire the people who are let go of; but rather should focus on the people who are already being utilized in other companies. And if your current staff can keep the projects flowing; it’s guaranteed that you’d be able to get good people to get them done.

So for the opportunists - I say “yes, we can”.

Escaped

Finally I'm going to be out from my current project, which is like the worst project I've ever worked on, not only because of the technology aspect of it but immensely because of the way the project is managed. People who work, has to work 24X7 without complaining and for the people who doesn't, there's no impact at all. I've already gotten myself in to enough trouble by speaking up against the way the things were managed so I'll refrain from going in to detail here. But the above picture tells the story.

From next month onwards I'd be on a new project, which I've been involved from the side for some time now. It's a banking application developed in .NET 2.0 and has some serious multi threading, remoting and webservices. With the involvement so far, management seems to be fairly good as well. So I'm looking forward for it now.

On your mark, get set, write code...

Usually, when I'm in a project, I pretty much get a good understanding of the domain and a thorough understanding of the requirements before I start to code. But this time, it's been a totally different story.
We are suppose to deliver a project on Friday and non of us in the team has a enough understanding of the domain or the requirements... we pretty much followed the instructions that the customer gave and wrote an ever changing set of requirements according to an ever changing design. It's an miracle that we made it this far and it would take more than an miracle for us to deliver this on time.
Basically most of us spent 16+ hour work days for the past few weeks because of this issue.
If some one ever ask me how a project should not be run, I'm experiencing the exact thing right now.
But after this, I'm going to be working in a cool new application involving more on architectural work... but my fingers are crosses :D

Prabath's last day

(Picture taken in 2006 September)

40 Months ago, on 1st of July 2004, Prabath and I started work at Virtusa. We both were in the same project for three years straight and passed so many milestones together.

Prabath is a person who can take on any challenge and overcome that in style. Every constraint is just another reason to be motivated for him. He always kept his head up and kept things rolling until he reached his targets.

He's a heartiest friend on whom I would trust my life with.

Today is his last day at Virtusa. His absence would be much felt in the years to come, as a top notch techie as well as a great friend.

I wish him all the best in his life...

Coding at last

After a frustrating period of creating documents that doesn't make any sense, I'm back to coding. The design isn't the best but I'm making the changes I feel like to make it more digestible.

When I go through a period of non coding I get so bored and frustrated and every thing gets so miserable. But starting to code is like seeing the sunshine again.

Lhotka’s theory is against the theory I understand

I’m so bored with the work that I have at the office so I’ve taken on to reading quite a lot. I started with WF and at the same time started to read “Expert C# 2005 Business Objects” by Rockford Lhotka.

In the first chapter itself, while describing business objects, Lhotka states that “So, a business object that represents an invoice will include not only the data pertaining to the invoice, but also the logic to calculate taxes and amounts due. The object should understand how to post itself to a ledger, and how to perform other accounting tasks that are required. …” (Second Edition – Indian reprint : page 23)

I disagree.

An object should be an abstract representation of an entity or a concept. An object should represent one and only one entity or a concept. The contracts the object may have with other objects should be strictly in need to have basis.

Lhotka states the invoice should contain the logic to calculate taxes. Tax is a concept independent of an invoice entity. The responsibility of invoice with regards to tax is only to support the following two functionalities. 1.) Provide the taxable values 2.) Accept the tax values and use them against the taxable values. The invoice should not even know who does the tax calculation. It should expose the above mentioned functionality in terms of events and methods in the public interface.

Lets take a brief look at the problems that would arise when we implement the tax concept inside the invoice. To start with, it’s difficult to manage. Every time the tax calculation methods are changed, or a new tax is introduced you have to change the code in your invoice and test every functionality that’s associated with the invoice. Secondly, think of security, tax calculation is likely to require more security than the invoice as it could be accessing financial resources, and invoice is an object you are likely to pass around in your application. Not a good combination to go together. And thirdly, this implementation require the invoice object to have contracts with other objects which are required in tax calculation, which further complicates the manageability and possibly impact the structure of the application.

The second thing is, Lhotka states that the invoice object should understand how to post itself to a ledger. It is acceptable for an invoice to know which ledger it is posted in, but to be able to post itself to a ledger, it needs the knowledge of the ledger before it’s actually posted to it. Which is conceptually wrong. The invoice in Lhotka’s example is having a contract to a ledger before it actually gets posted to it. Secondly, posting an invoice to the ledger is neither a responsibility of an invoice nor of a ledger. An invoice should support being posted in a ledger and perhaps hold a reference to the ledger after it’s posted. And the ledger also should support posting invoices to it. But the responsibility of actually doing the act of posting is a responsibility of a third object.

Having such unnecessary contracts and responsibilities in objects causes manageability issues as well as design issues further on. For example if the action of posting an invoice to a ledger becomes more complicated in the future that requires modifications to the invoice class. And if these changes require access to other objects which the invoice by design does not have access to, it’s going to require some risky structural changes to the application as well.

Any thoughts?

Starting it all over

It's been some time since I moved in to a place with fixed line telephone and my office firewall unblocked blogger. So I could have really blogged earlier than this if I wanted to... But I didn't feel like it up till now.

Things have been generally okay but could have been much better. To put aside personal life, which again could have been much better, work has been as boring and frustrating as it could be. I enjoy solving design puzzles and writing code. But that's exactly what I don't get to do these days.

The company is more focused on engaging in enterprise level business even at the stake of not following proper engineering practices, and that conflicts with my interests.

Yasas interviewed

Yasas is a good friend of mine who started his career at Virtusa in the same project with me. Soon he opted to change his career track to BA to become undoubtedly one of the best business analysts in the business.

Recently Debbie Timmins of ACS interviewed him in her blog. You could read the interview here.

You could also visit Yasas’s blog here.

Choice

Four years back, .NET 1.0 was relatively new. I had started to read and play with it from the beta so I was pretty good when the 1.0 was out. When I started professional development in mid 2003 I was on top of the business.

One of the luxuries in smaller companies is that you get to work with what you want - at least most of the time. That allows you to use the latest technology in day today work. But in large IT service companies that’s not the case. Most of the time they take over upgrade or maintenance work.

So even though I managed to keep myself up to date after joining Virtusa, my work had always been in older technology; but it had at least been .NET. By the time .NET 3 was released, at work I was just migrating a .NET 1.1 code base in to 2.0.

But now it seems as if things are to be worse. There’s VB6 maintenance project in the horizon signalling to come my way…

Tell what you will do, do what you told you would do, say so if you couldn’t

Many software companies use fairly matured and evolving project delivery processes. Most of those processes could be tailored to suit individual projects. But we often see projects that runs in to misery, and worse, there are projects that have 100% process compliance and still runs in to misery - and some times fails!

The main cause as I see is project managers think process is like law. So if they fail to follow it, they cheat. They lie, produce forged documents and do every thing in their ability to mislead the process auditors. In this process not only they jeopardize the project, but burn out the resources and risk demolishing the moral of the team members. They do not realize that it’s not something measuring their competency or performance but a simple guideline to aid their work.

Lying to a process auditor is like lying to a doctor. No matter how humiliating it is, you got to tell the truth. In a service oriented company, delivering a quality service on time is the key to good business. And a process is there to guide that delivery. If things go out of track, they need to be identified and rectified.

If I blog about this...

I was accused of blogging about politics and terrorism just to attract traffic. No hard feelings - I'm excited to see more traffic, but I write what ever that comes to my mind when ever I feel like writing it down. When people read them and comment, there's a little excitement in it. But I don't think what I write about politics or the terrorists would have any impact on the system or the situation in the country. Nor do I expect to change the way people think. It would be bonus if they ever do.
There had been many a discussions in this blog on the terrorist issue (No it's not ethnic and that's my stance) and how it should be handled. Every one had their say and then that's that.
The current news highlights are on eviction of Tamils who are from NE and could not provide a viable reason for staying in Colombo from the lodges. I'm tempted to write my view on it, even when it serves no purpose.

• Why were they here in the first place? Why did they left their homes to come live in lodges in Colombo?
• How on earth do they afford paying for lodging, food and all other expenses? They do not work here. Can't presume they earned that much money while in NE.
• What do they do here? If they aren't working or learning, why are they in Colombo for months? How and on what do they spend their time?

Anyways, that's that and I'm having a issue with how Microsoft Resource Schema 1.3 and 2.0 stores OcxState. There's a difference but I can't find it documented any where...

The rights of terrorist suspects in Britain

Gordon Brown, who is to succeed Tony Blair as the next British prime minister, is proposing to triple the length of pre-charge detention period for terror suspects. So that's going to be 90 days. And he also plans to make amendments to the laws so phone tap evidence can be used in court.

search google for more

Human rights organizations again seems to have found a bone to byte on. I won't blame them, they need funds to keep the organization and they wouldn't receive funds unless they oppose a government.

But what we must understand is that every country who are facing terrorist threats are forced to make such compromises which indeed would violate human rights to ensure national security. It's about safeguarding the lives of the people in a country.