Why stretch goals are stupid

In case there was any doubt from the title, "stretch goals" are idiotic, wasteful, and counter productive. This company I'm at has been using them and I find them to be a reprehensible practice.

Stretch Goals discourage accurate estimation
By their very nature stretch goals are based on the assumption that the team has excess capacity that is available to meet more goals than those firmly committed to. Because of the value traditional project managers place on being able to "stretch" and do more work than was initially estimated, this will become a self-fulfilling prophesy. Team members will start to pad their estimates in order to make stretch goals practical. Inaccurate estimation ruins a team's ability to track velocity.

Stretch Goals reduce overall quality
When team members complete their minimal level of tasks and still have time in a sprint, they often will refocus on current tasks in a new manner, like adding additional test automation, code refactoring, or design reviews. The presence of a stretch goal pre-empts this good behavior by instead asking team members to take this extra time and apply it to a new (and likely unreachable) goal. In extreme cases the team will start cutting corners on mainline features (which are higher priority) in order to begin work on stretch goals.

Stretch Goals reduce team ownership of tasks
What I have witnessed here is that in having the scrum master press for stretch goals every sprint, the team stops acting as if it owns the tasks and instead goes back to a traditional project management-engineer type relationship. Since the story commitments a team makes are being constantly overruled by the addition of a stretch goal, team members start to change their behavior to the point where they meekly wait to be told what to do instead of actively participating in the planning process. End result: devolution into mini-waterfall behavior.

Evaluating Source Control Systems

Or the other title - TFS Sucks Donkey Balls.
Normally I don't rave. I rant first, then rave. My first encounter with Microsoft's latest foray into whole product lifecycle management was as an unbiased review for moving my former employer forward off of Visual SourceSafe onto a new source control system. Well, not totally unbiased because SharePoint sucked ass at the time. Now I understand it sucks less ass in its current form, but still there is some ass suckitude. But I digress. Basically, TFS was a pig, required an entire dedicated machine, was slow as molasses in January, and all the whoop-de-do features like check-in policies and integrated unit testing and code coverage required a different, more expensive, and bloated version of Visual Studio. It also took me two or three tries to get it to install in a reasonably sustainable configuration.

Contrast that to Subversion which was very quick to setup, unobtrusive and required no additional hardware. We did later take a couple days to put it on Linux/Apache for stronger authentication and safer access, but that was not required to use the system (clearly, we went with Subversion - and never regretted it).

Here is an actually well balanced comparison of the two, while here is a Microsoft fan-boy comparison, if you don't want to hear me rave any more. My comparison grid is lower in this posting.

Now, a little over a year later I'm at a company that's using Team Foundation Server. You know what - its even more awful in a production environment than I thought it would be from my testing. Here is a relatively short list of complaints:

• It is slow. And I mean sloooooooow
• Merge tools are second rate
• It forces auto-checkout on you, even when configured with confirmation (just like VSS did)
• You can't share files among projects, and likely never will be able to do this without a 3rd party solution
• It performs phantom mystery checkouts of files unrelated to what you are doing
  
• If your TFS server goes down your developers will be all but unable to even open their solutions and do work
• TFS is unable to detect a changed file on checkin unless that file was explicitly checked out
• Non-Visual Studio users will be unable to make use of your TFS source repository (ex: designers)
• Just opening a solution causes TFS to reconnect to the server and attempt to refresh its file list - AND YOU HAVE NO SAY IN THE MATTER
• If a file gets deleted outside the Visual Studio environment, TFS will be unable to recognize this and a Get Latest operation will not recover the missing file.
  

Why is all of this true? Wasn't TFS a sign of Microsoft's recognition that VSS is, to put it mildly, and inadequate solution? Maybe originally, but then they placed the same joker who was in charge of VSS in charge of the TFS project. As a result, many of the same technical problems and ALL (plus some) of the workflow and usability problems have re-surfaced in Team Foundation Server.

It is also true that TFS is almost unusable because Microsoft doesn't dogfood this product (the irony of this reference is that Microsoft is cited in the first line). Internally they use something called "Source Depot", which sources tell me is actually a custom build of Perforce.

Is it all bad?
No. They did fix a couple of things. Your source control database will no longer randomly corrupt, and shared source editing is fairly reliable (although the merge tool is of very low usefulness). Checkin policies are a nice addition, tho hardly the earth shattering feature Microsoft has made them out to be. Changeset numbering is quite convenient. Shelvesets are also a nice feature - basically a mini-branch.

Show me the TFS vs. Subversion Comparison Grid
You asked for it.

Feature	TFS	Subversion
Binary file support	Yes	Yes
Revision History	Yes	Yes
Checkin rollback	Yes	Yes
Source Branching	Yes	Yes
Source Merging	Poor	Yes
Pluggable Merge Tools	No	Yes
IDE Client	Yes	Yes
Standalone Client	No	Yes*
Command Line Client	Poor	Yes
Checkin Policies	Yes	Thru custom dev
Bug/work item tracking itegration	Yes +	Yes ++
Multiple Workspaces	Yes	Yes
Portable Workspaces	No	Yes
Robust code recovery of local workspaces	No	Yes
Database storage	Yes	Yes**
Filesystem storage	No	Yes**
Mixed development environment support	No ***	Yes
Distributed/Public Network Support	No	Yes (on Apache)
Offline/Disconnected Support	No	Yes
Server Cost	$$$	None
Seat Cost	$$$	None

Notes:

* With (usually free) third party tools

** Subversion supports Berkely DB storage or filesystem storage - filesystem is usually recommended

*** TFS requires a Visual Studio project/solution for many basic operations. Sometimes you can fool it

+ TFS works out of the box with it's internal bug/work item system (crummy spreadsheet backed). Third party integrations coming to market slowly

++ Many, if not most, major bug tracking and project management systems already have Subversion integrations in place

Cards vs. Tools

Well, that startup collapsed. Under-funded. I had a few things to sort out in the mean time, hence the long gap. As I turns out, I got a sweet gig as an Architect for a major Internet site (who shall remain nameless). They do scrum over here, but quite a different variant than I have been doing.

The biggest difference is the use of cards/sticky notes for story management instead of a tool. I have to say, I don't like it. The biggest issue with cards is that there is a complete loss of historical data. Once the card goes into the completed stack or is torn up all knowledge goes with it. This includes sprint in which it was completed, original story points, issues, whether is was later unfolded, actual effort expended. There is only a velocity tracking for the last two sprints, and it is somewhat suspect.

I've also noticed that our backlog - perhaps subconsciously - has been limited by the size of our board and the number of sticky pads that will fit. It is actually impossible to retain even two months of data (forward or back), much less six to 12 months of data. Ouch.

While the tactilness of cards is nice during the sprint planning game, the loss of historical data far outweighs this. Modern Agile project management tools allow for drag and drop, so you can get this interactivity by using a laptop and a projector.

Rails - The Good, The Bad, and The Ugly

Before we start, so you all know where I'm coming from and don't think I'm picking on anyone, I'll give you the following summary of other web frameworks out there (and this is all first-hand experience):

• Asp.Net: C# is a great language, but Postback controls suck balls. In fact the entire Postback model is bankrupt and should be thrown in the garbage.
• JEE: Overly and unnecessarily complex for 95% of real world problems. Pain in the ass to get a new environment up and running.
• Java/[framework of choice]: Too many options and starting to show it's age.
• PHP: Just a bad and buggy implementation of classic ASP.
• ColdFusion: ColdFusion is a sick joke. Use this only if you don't know how to write code and like server crashes.
  

In the course of this you will read some things that are not due to Rails per-se, but are something Rails inherits from Ruby. To anyone who complains about this (as I've read in the comments of other posts), I say "tough titties". When you pick a language you inherit all it offers - for better or worse. Any issues that come back to Ruby are also your issues, Rails. Just the same way that all the cool things Ruby offers you also get some credit for.

OK, so now let's talk about Ruby on Rails.

The Good
Based on my analysis of the Rails framework and using it, this is a framework that is optimized for single developers or (very) small teams. I say this because it provides tools to build all layers of the application (data, middle/server, and client) from within the Rails environment. It is built to be your everything for building an application.

Rails is great for prototyping a system or for bringing someone who's new to web development up to speed. It has a tightly structured framework that is well thought out. It also promotes good coding practices via the integrated testing framework.

The design of the RESTful API and routing are top-notch. It is also the first web framework I've seen that provides clean and tight integration hooks for client-side javascript libraries. In every other platform I've used the client side hooks are bolted on as an after though. And they feel like it.

Rails provides a console for dynamically testing the objects. Just create one, invoke it's methods, and have fun. In fairness, .Net provides this with the kick-ass new PowerShell, but doing things this way is not common practice (tho we probably should).

You get gems, which is a kick-ass Ruby thing for installing and updating all sorts of components. Picture Eclipse's add-on functionality, but without the bother of constantly having to manually add new repositories. Find a new cool tool you like, type $> gem install watir.

Rails Engines are a totally sweet idea. Instead of using wizard builders for functional blocks like Visual Studio does, you install and configure whole functional blocks -> user logins, etc.

The Bad
Rails is built to be your everything for building an application. This means it does some things fairly to exceedingly well, and other things exceedingly poorly. Unfortunately, since it is a tight stack, Rails will fight you if you try to use another tool/technique for an aspect of your application.

What does that actually mean?

Well, for starters it tries to be smarter than you for common tasks. Often it is, but your code ends up inefficient. The has-many :XXX / belongs_to :YYY makes it quick to wire object data relationships up, but it leads to tremendously inefficient code. Common reference tables wired up this way will be constantly queried as you use the objects.

Rails treats the database engine as a second class citizen. This may seem counter-intuitive to many people who have read some of the Rails docs about "why ActiveRecord" or are relatively new to ORM frameworks. And it is. Counter-intuitive. That Rails doesn't leverage the database engine in a very intelligent way. The bulk of the meta-data is ignored, or worse, discarded leading to - what's the phrase... - shitty database design.

OK, so what do I mean? Well, if you use the Rails format for managing database creation all of your foreign keys will be thrown away. Data integrity anyone?

Migrations are a nice idea. In practice they suck balls.
Let me see if I can explain it in layman's terms. Problem: Database design changes with the growth/implementation of a project, requiring developers to sync their data models.
Rails Solution: Auto-gen database differences and prompt devs to apply them over the course of the development cycle. Real Solution: Be able to rebuilt/redeploy database at any point from script, both in new state and in "since last checkpoint" state. The problem with migrations is that they are not just a "least common denominator" solution for supported databases, they are a subset of the least common denominator (ever heard of a bit data type?).

This is an immature platform. No great IDEs are available (although Aptana/RadRails is pretty good). Oh, and to anyone who says IDEs and the people who use them suck I can only assume you are joking or have never worked on an application of much complexity or size. Try managing 250,000 lines of code with vi. The "Rails uses fewer LOC" argument only goes so far - you don't see over 90% code shrinkage. And those benefits are greatly exaggerated as well. You end up giving back much of the gain in the form of increased testing required by the lack of a compiler and type safety.


The Ugly
Ruby on Rails is not thread safe and runs single threaded. WTF!!?!?!?!?!?! I just about crapped myself when I heard this. This alone is enough reason to never, ever, ever, ... ever write anything other than a prototype or toy app in RoR. Hopefully this will be addressed soon, but it's doubtful given some of the comments in DHH's blog. Who in their right fucking mind would write a single threaded web framework. Anyone remember why Session was so evil in classic ASP? It's because it had to run single threaded. Oh, and Rails community, I have yet to see a problem solved with a single thread that cannot be solved with multiple theads and critical sections.

Check out this informative slide show on how to scale Rails. Basically they have created a platform that cuts your coding down by 20-30%, but increases you deployment and maintenance by 80-150%. As many studies have show, lifecycle costs of any software system are predominantly in the maintenance, whereas initial development costs usually account for only 30% of a a system's total lifetime costs.

The Rails community seems to be largely composed of ex-PHP developers. As a result you have a dearth of trained computer scientists. This results in asinine solutions to complicated problems. Not that you don't see dipshit implementations of things in every single other framework on the planet, it's just that these sub-optimal designs don't become part of the respected lexicon.

As a final note all this fawning over David Heinemeier Hansson (the initial creator of Rails) is a bit unseemly. I'm all for PDAs of hot man-love, but usually I expect to see it outside The Cuff on a Thursday night, not in some drunken rant-blog surreptitiously about programming.

A Shining Ruby

Yes, so this company is all hot to trot on Ruby on Rails. Or rather, the CTO is. I can sum up my experience to date as follows:

Stage 1: Loved It
Stage 2: Hated It
Stage 3: Tolerating It

I'll do a longer post in a little bit, but here's the preview skinny.

Stage 1: Loved It
Ruby on Rails is a very smart framework. Ruby _is_ really a fun language to use. Ruby is great for little utilities and quick scripts and has all sorts of interesting things you can do with dynamic code. Think Javascript with a little more sense to it. Rails takes many of the best techniques from web apps, including Test Driven Development, and rolls them into a single stack (cross-platform).

Stage 2: Hated It
This stage came after actively building an app for our first sprint (two weeks). Like many tools that try to help you out, it will fight you if you try to go outside the prescribed path. Once I was actually building with it I realized it was more like doing battle with the ghosts of classic ASP/VB Script than using some new whiz-bang tool. And once you get beyond the cutesy tutorials, the documentation truely blows when compared to what can be found for other, more mature platforms. And no, being able to read the source code directly doesn't preclude the need for proper documentation and tutorials.

Stage 3: Tolerating It
Like any platform, there is a learning curve. I'm accepting that I'm still going up it. On the positive, it is much easier to learn that a full Java stack. On the order of a couple days instead of a few weeks. Unfortunately, the performance (not scaling) of Rails is so poor that I end up writing all kinds of workarounds in addition to extra unit tests that would not be necessary if I was still developing in .Net with a proper ORM tool. This largely cancels out any perceived benefits or LOC savings that people crow about when evangelizing for Rails.

I'll follow up with a full review of the good, the bad, and the ugly of Ruby on Rails in a few days/weeks once I confirm a few more things.

Scrum in the Small

So, in the new gig there are three of us developing and one designer. And an off-site contractor-guy that we don't talk about. And multiple applications, so everyone pretty much gets their own piece. Classic startup. Do we even need a process?

Emphatically, yes.

Startup stage companies face a different set of problems. Organizational inefficiency is not one of them. There is not bureaucracy to slow you down. There are not so many people that coordination is difficult. But efficiency is still a concern.

Most startups face both the "too much work" and the "shifting priorities" problems. Scrum helps with both of these by placing everything into a single priority queue. We opted for one across all products. The backlog also allows for management of shifting priorities. As a bonus, if you track your velocity you will have a good idea of when the product will be ready to ship.

Thankfully there was little resistance (none) to implementing Scrum here. There is a little uncertainty as to if it's actually needed. As my sales pitch I put it that scrum is essentially exactly how we are working anyway (read: little to no overhead), but helps us prioritize and focus.

We're now in our second cycle. Big winner points:

• Backlog (priority queue) for tracking progress
• Planning game (story point estimation)
• Sprint Demo to stakeholders
  

Items that I see as having value, but not yet proven to team:

• Velocity tracking (and long-term release planning)
• Daily standup
  

The standup is interesting. In startups where we all sit together there is a perception that the standup is not necessary since we're already communicating. This is a false perception. We actually have almost no communication since everyone is heads-down on their part. The standup brings forward where our touch points lie.

Moving on and up

It's been a little while since I've posted. Things have been fairly shaken up, which has kept me busy. I reached a point where I realized it was time to move on from this initial experiment. We had a project manager who had picked up the Scrum grail. The company was far more comfortable with someone in that role instead of a developer driving a project management methodology.

While it is healthier for the company organizationally and politically, I realized I would no longer be able to affect the sort of changes and improvements I have come to expect of myself. Don't despair, tho. I've started with a new pre-product startup and will be continuing on, albeit sporadically and with a different slant.