Dude! Where’s My Backlog? The People and Process of Product Ownership.

  • 1

Dude! Where’s My Backlog? The People and Process of Product Ownership.

Tags : 

In September 2010 I gave a keynote talk at Agile Eastern Europe called “Dude, where’s my backlog?” The talk covers the common pitfalls seen with Product Owners and the Product Backlog. The talk covers the people that bring a product backlog into existence and the processes they can use to maintain it. This includes using ideas from Kanban for personal process and modifications to Scrum for more frequent involvement of the team in maintaining the backlog.

Robin Dymond Keynote: “Dude, where’s my backlog?” from Agile Eastern Europe 2010

  • -

Iterative User Interface Design in 1991

Tags : 

QSystem: realtime 3D audio for recording applications

QSystem: realtime 3D audio for recording applications

In 1990 we created QSystem, the first 3D audio system for professional audio recording applications. The system was used by Sting, Madonna, Roger Waters, Pink Floyd, and many other artists in the recording studio. Unlike surround sound systems today, this system created audio for playback on 2 speakers. The sound was positioned using filters that tricked the mind into perceiving the sound came from beyond the stereo speakers. The version 1.0 and 1.1 of the system had 8 channels of position processing, and the locations were fixed in a 180 degree arc. In QSystem version 2, pictured above, we wanted to add the ability to control the position of any channel with a joystick, and to record joystick motion so that positioning would be played back with the recording. This meant a completely new user interface, a custom joystick, many new system software features (C++ on DOS) for handling the joystick, recording, and interfacing with the studio automation system, and changing the core audio digital signal processing architecture code that ran on the Digital Signal Processors (DSPs). We also designed a new DSP processor board at the same time. All this with less than 10 people.

The User Interface challenges were significant. We had a system whose primary purpose was audio output. We had to provide a real time graphics display for positional information of each channel, but this was DOS on a 386 and CPU was needed for audio processing control. The processing that did the audio positioning was non-linear and constrained the movement paths within the semicircle shaped sound field. We also had a lot of uncertainty since there was no product like this on the market. When faced with a problem like this we had the benefit of beginner’s mind. We had no idea what was going to work, so we started trying things and kept it simple. I worked on the DSP algorithm, looking for a way to smoothly control audio with a joystick. An industrial designer worked on the joystick case. Our best programmer tackled the joystick recording, playback, and synchronization features.

Some stakeholders drew complicated 3D pictures of how the UI should look, including the founder. Many discussions ensued where we had to explain that we could not commit to such a design until we knew more. As we built up the features, we created a simple 2D map of the speakers and semi circle sound field. The position of the joystick was a dot on this map. As we played with the UI we found that the sound position was not reflected correctly in the UI, so we changed the parameters controlled by the joystick to improve the auditory realism. Once this was working the recording and playback features were added to the system, again the UI was simple, driven mostly by the buttons on the joystick.

While we were working on the system others searched for alternative displays to a bulky PC CRT monitor. The CRT was heavy, not very rugged, and as we discovered with QSystem v1 they caused shipping problems. Late in development the team found an industrial display that met all our system needs, however it was a monochrome LCD. We tried it out and it worked great. Because we had not invested time in a complicated interface we were able to make this robust but simple display work easily.

During testing in the studio the founder and others who had initially wanted the complicated interface got a chance to use the system for hours. In use they found the interface’s simple design was more than enough. They liked the way it focused the attention on the audio instead of the screen. With most of the functions controlled by the joystick, the sound engineer could forget the display and focus on the sound. In later revisions we made some improvements based on feedback. The complicated 3D interface came up a few times and we thought how misguided an idea it seemed knowing what we now knew.

We were successful in solving a complicated UI problem in an elegant way by moving the technology and UI forward together. The technology imposed constraints on the UI, changing our initial design ideas. The joystick made us rethink how to control the system to make it easier to use. The late arrival of a major UI change, the monochrome LCD, did not cause major changes to the UI since we had kept things simple. A complicated graphics intensive UI was strongly advocated in the early phases of the project however it turns out that it would have been a disadvantage in effort/cost to implement, cost of CPU resources, and would not have worked on the monochrome LCD. Locking in complicated UI design upfront would have significantly increased the risk of failure and delays in this project. By focusing on simple solutions as the underlying system was developed, we arrived at a solution that was simple to use, task focused, and flexible.

  • 3

Agile User Interface Design and Information Architecture from the trenches

Tags : 

I was a technology Director in a large web design company 6 years ago, and they failed to adopt Scrum. There were numerous management dysfunctions, however the Creative managers were the most resistant to change. Often this resistance was a case the creatives not wanting the reality of building enterprise software to hinder the design concepts they were making in Photoshop or Illustrator. When these design concepts met the reality of enterprise architectures on the development floor there were major implementation problems. This dysfunction, along with high pressure, unrealistic delivery dates and long hours resulted in missed deadlines, high stress, low code quality, and 40% annual turnover in IT staff. Being at this company for a short time spurred my interest in User Interaction design (UX) and Agile, and I have since found basic solutions that work when people want to do UX and Scrum. To date (2011) I would say that of all the disciplines needed for creating software and especially web sites, people coming from UX background are often the slowest and most resistant to adopting Agile principles and practices. I think this comes in part from the influence of Allan Cooper’s book “The inmates are running the asylum.” However Allan Cooper, himself a practitioner looking for solutions that work, embraced Agile and Scrum since 2008, if not earlier.

Here is the way I work with UI Designers and Information Architects who want to work Agile:

    1. The Product backlog and its priorities drives all the work. We work from business priorities not UI priorities.
    2. Sketch the highest level level UI for the site. No drill downs into buy flows, just top level.
    3. Look at the backlog and start thinking about what the team will need for UI half an iteration ahead.
    4. Make paper prototypes (sketches, paper menus, paper buttons) to support upcoming user stories for next sprint. Don’t embellish with new features the designer thinks are needed. KISS. Include validation and other acceptance criteria for data fields.
    5. Create and maintain a style guide and reference it as required. Create and use reference page templates as required.
    6. Combine 2,3,4 above into a 1 page design spec. to support the UI needs of a user story.
    5. Review prototypes with PO, QA and lead dev a few days before sprint planning, ensure they think it is good enough and it is testable. Keep notes regarding potential trade offs.
    5a. Check in design docs, version them.
    6. Participate in sprint planing with rest of team.
    7. Work with team on implementation, clarify details of design (dimensions, locations, behavior, etc.). Work with QA on test plans for new UI. Help build UI (HMTL/CSS/PHP/Javascript etc) in dev IDE, pair with dev as they wire it up. Pair with QA to run test cases and implement automated UI testing (see tools like Rspec, Cucumber, etc).
    8. repeat for every sprint.

To recap the UI designer starts half a sprint ahead of the team, helps the Product Owner with UI issues, and creates very lightweight and flexible prototypes as input to sprint planning. The rest of the work is done within the sprint. I have found this is “just enough” lead time to have thought through UI issues without creating solutions in a vacuum away from the team.

Additional tactics:

  • Create a style guide for the team and have them read it, listen to their feedback and continuously improve it.
  • Create templates: every page is not a unique creation. Refer to those templates in prototypes and design spec.
  • Break dependencies between UI/layout and business logic at beginning of the iteration. Agree on data/fields/controls at beginning so business logic can be rendered to a very simple layout free page.
  • Test your paper prototypes with users by asking them to do certain operations and then observing the resulting actions. Don’t worry about big usability testing at end, do it often and informally with people who are easy to access and know enough about that business.
  • bad idea ui tool

  • AVOID COMPLEX HIGH FIDELITY USER INTERFACE DESIGN TOOLS. They really slow you down and lock in decisions far too early. These decision are often wrong.

Lack of detailed knowledge about the customer needs and technical limitations can result in an unstable UI. This lack of understanding results in an unclear product backlog and user stories coming into the sprint. To cover these gaps the UI person has to go find out the missing details to create the design, which slows them down and results in numerous design changes during the sprint. The UI designer is making up for lack of knowledge or lack of effort on the part of the Product Owner. Fix this root cause by getting the Product Owner to improve the product backlog so the designer and the team have better user stories and acceptance criteria coming into the sprint. User Interface work is simply another form of software requirements, as are software architecture, features, and test plans. All of these activities can be done in a “just in time” and emergent fashion and result in consistent architectures and usable interfaces. As User Interaction designers it is important to focus on the end result, great software, and to help the team realize that goal. The UI design artifacts: wireframes, clickable concepts, specs, etc. are often over-valued by UI designers. These are tools in our toolbox and are not essential. Choose the tools and the collaboration model that takes the least amount of time and offer the greatest impact on the goal of delivering great software. That goal is best achieved by working side by side with team members building the software. Many are doing it today. However it does require people to change their behavior, and that is likely the hardest thing to do.

  • 1

Wall Street Journal? New York Times? Why is Lean and Agile not a story?

Tags : 

Over the last 2 years we’ve seen some pretty hard times in economies in Europe and especially the USA. Financial services and lending business were in an state of near collapse. With each day companies were announcing job cuts. Job losses in the US are higher than they have been in 30 years. Now that the economy has stopped shrinking, and markets are starting to improve, companies are facing a much more difficult business situation. Stock prices and profits have recovered in some sectors, however most companies face slow growth, and systemic changes in their marketplaces. The US consumer has lost a substantial amount of net worth. That group is not coming roaring back to consume like they have in the past. Many of the businesses that cut staff will not be hiring them back, either because they don’t need the capacity, are doing something different, or are out of business.

Many State and local governments are facing heavy shortfalls in revenue. One important reason for declining tax revenue is property taxes. Property taxes are hard to collect when the property is in foreclosure. Commercial properties are also strapped as their clients break leases or only make limited payments. These losses often result in commercial property owners going out of business. The commercial property failures significantly lag consumer mortgage foreclosures, since commercial properties are not affected until their clients start going out of business. Booming government deficits, high unemployment, slow growth, and increased competition in a global marketplace.

As bad as the economy is now, I am sure it is not as bad as Japan’s collapsed economy just after World War Two. That is when Toyota decided to enter the automotive business. 60 years later Toyota leads the industry by most benchmarks, and are more profitable then the next 4 automotive companies combined. During this recession GM and Chrysler collapsed, and all three big auto companies required billions of taxpayer dollars just to survive as much smaller companies. At the same time Toyota took thousands of idled workers and launched retraining programs so they would be ready for the next generation of manufacturing, new assembly systems, and upcoming products.

The great recession of 2008/9 was bad for airlines, many shrank 10 to 15% and some went bankrupt. During this same recession Southwest Airlines managed to grow by 1%, while paying its pilots more than any other US Carrier. Southwest has been profitable for 36 years straight.

There is a major story that is being completely overlooked by the American Media. That story is how some companies have found a way to gain not one or two years of competitive advantage, but 10-20 years competitive advantage. How some companies consistently innovate and thrive, while others consistently fail to meet the expectations of their customers, employees, investors, and the public. These under-performing companies may have survived the recession, however unlike GM and the banks, there is no bailout coming from cash strapped governments to save them.

What do successful companies like Toyota and Southwest airlines do differently? Can they provide a road map for other organizations? What stopped GM from changing and how to recognize those patterns in other companies?

The New York Times and Wall Street Journal are missing the boat for their readers. There is an important story that European and American business leaders are largely clueless on. I know because of the conversations I have with their VPs, managers and staff. The executives running those companies have a fiduciary duty to get up to speed and start to redesign their organizations so that their companies stop surviving and start thriving.


  • -

Oracle, databases, Scrum and Agile

Tags : 

I recently read a great post by Cary Millsap, an influential member of the Oracle community on Oracle and Agile. I have read both of the books Cary mentioned, they are excellent, and the Goal by Eli Goldratt is outstanding, if you haven’t read it, put it at the top of your list. It is a novel and has nothing to do with software.

I don’t really understand why Agile is not embraced by the Oracle community. There are lots of reasons why it can make a developer’s life better, and it doesn’t matter whether it is coding SQL or Java. However from a DBA perspective, I think having changes committed to production DBs every week or two is scary, especially if you are the one carrying the pager and responding to 4 AM outages (and why do they seem to mostly occur at 4 am?).

For Agile to be successful we need to deploy early and often. For production stability and performance we need to quantitatively know deployments are going to work just as well or better than the last deployment. There need to be safeguards put in place to address the risks, and systems that reduce or eliminate the transaction costs of doing deployments. As mentioned in Cary’s post, automated testing plays a huge role here.

Here’s one way of stating the DBA’s needs in the form of a user story:

As a DBA I want to have a stable, high quality database to monitor performance and tune so that I can maintain data operations and improve system performance.

Acceptance Criteria:

  • New database releases to production must contain all the performance improvements implemented in production to date.
  • All DB changes must have automated test coverage that quantify whether or not the DB changes implement the desire inputs and outputs correctly
  • Any DB change found to not function correctly can be rolled back without impacting the rest of the DB
  • All changes between production deployments are automatically calculated and displayed to DBAs
  • Production Databases are stable and not changing between the regular release cycle.
  • DBAs collaborate with and are consulted by the team(s) implementing DB changes
  • DBAs provide weekly mentoring and DB code reviews with team members implementing changes
  • (Please add your additional acceptance criteria)

What changes you would need to implement in your organization so that the above user story were true?

If you are a DBA working in a small company next to developers and QA, then maybe it is changing a few of your practices, working more closely with the other developers on DB changes, implementing test frameworks like DB Unit, getting SQL code into version control, implementing a DB change management tool like Liquibase, and creating a better rollback strategy.

If you are in a large bank sitting with 15 other DBAs at the backend of a manually intensive, form driven, cost managed, outsourced deployment process, you have some work to do :). The goal and many of the steps are the same as with the small company. Ironically, the bank has many more resources to implement these changes. However the management hierarchy and organizational silos make it much slower and more difficult to implement these changes. So what do you do? Well you can continue to stay in your DB team silo, complain that Agile is the problem and be the victim. This means that most interactions with others wanting changes will be negative, you will be viewed as obstructionist, and that cute tester will think you’re a jerk. Another approach would be to remember that we are all on the same team, and go for a walk and meet the Agile teams to see how you can work together. You might take this blog and the user story above, show it to the ScrumMaster, Product Owner and Developers. Start a friendly discussion about how your needs for a reliable, production friendly DB can be met in the context of regular production releases. Remember that by doing this you are asking them to change as well, and since you are both changing, the situation becomes problem solving together. One more point, expect mistakes to be made and occasional firefighting on the road to a more effective development and database operations process. The changes are non-trivial, the results can be amazing, and the journey is the most memorable part of the experience. Bon Voyage!

Thanks to James Thomson for the inspiration on this one.

  • -

Lean Six Sigma: Are DMAIC Demons Possessing Lean?

Tags : 

Halloween is a few weeks away, and the strange world of the dead, undead, partially dead, or dead fashion is inhabiting the shopping malls. Over the last two years I’ve watched another kind of possession has happened in business thinking. Once the darling of Motorola, GE, Bank of America and others Six Sigma has been falling out of favor. Six Sigma’s DMAIC model, its emphasis on gathering statistical data and focus on elimination of variation has floundered in knowledge work. Knowledge work has at its heart, people. There is no machine stamping parts. In knowledge work, the work being done is continuously varying in size and complexity. The work of problem solving is done in people’s brains, and the effective flow of information in the work system controls errors. All of the variation in these complex adaptive systems means trying to eliminate variation is a zombie’s errand. Lean and the Toyota Production System are principles based. Many of the ideas in Lean can be applied to Knowledge Work. Ideas like Respect for people, Pull, Flow, Visual Management, Value Stream Mapping, Systems Thinking and Continuous improvement all make sense in Knowledge Work. These Lean and TPS ideas can be used with systems that adapt and change based on the variation inherent in the work.

Over the last few years Six Sigma consultants have started to re-brand themselves as Lean Six Sigma consultants. The adoption of Lean onto their business cards is partly a survival strategy. If Six Sigma’s not selling and Lean is, then let’s “do” Lean too. The problem is that Lean and TPS come from a very different perspective on work than Six Sigma. For example Six Sigma’s DMAIC process is:

  • Define the problem, the voice of the customer, and the project goals, specifically.
  • Measure key aspects of the current process and collect relevant data.
  • Analyze the data to investigate and verify cause-and-effect relationships. Determine what the relationships are, and attempt to ensure that all factors have been considered.
  • Improve or optimize the current process based upon data analysis using techniques such as design of experiments, poka yoke or mistake proofing, and standard work to create a new, future state process. Set up pilot runs to establish process capability.
  • Control the future state process to ensure that any deviations from target are corrected before they result in defects.

There are some major problems with the DMAIC approach when dealing with knowledge work:

  • Defining the problem with voice of the customer is great. However what if the customer can’t articulate what they want? Every software customer I have worked with did not have a functional crystal ball. They can often describe at a high level their needs, but they do not yet have enough information. The only certainty is that they won’t have enough information. Neither do developers.
  • The underlying assumption of Six Sigma is that the process under study is a repeatable process producing the same type of work product and subject to some statistically relevant variance. Measuring aspects of the process gives data to better understand where variation occurs. In Knowledge Work such as software development or marketing campaign creation the work is continuously varying so the measurements measure a complex adaptive system, not a repeatable process.
  • Analyzing the data gathered will provide some insights. Let’s say that data reveals a problem and a solution is found. Putting an improvement in place will definitely help. We may rarely do that type of work again. What happens when down the line a different problem occurs that we have never seen and do not have a measurement for?
  • How do we know we have the most correct process just from analyzing work in our current process? In Knowledge work, the work itself often impacts the process.

Six Sigma’s underlying assumptions fail to account for complex adaptive work systems that rely on people to do the problem solving and rely on information flow to do error correction. So why do people apply Six Sigma processes to knowledge work and what happens when they do? More on this later…. time to catch a plane.

  • -

Kangaroo boxing: Scrum vs. Kanban.

Tags : 

Software development has its trends and its innovations. The term Agile came about because people practicing different iterative workstyles decided to come together and develop a brand, Agile, that represents their common principles and values. Agile workstyles include Scrum, eXtreme Programming (XP), Crystal Clear, Feature Driven Development (FDD), and Dynamic Systems Development Method DSDM. Then people like Mary and Tom Poppendeick recognized that Agile software development could benefit from the ideas of the Toyota Production System, also called Lean for western implementations. Now some of the proponents of Lean, many of them from the Agile community, have been attacking Agile methods like Scrum, claiming that Lean is “better.” To me this doesn’t make sense. Agile and Lean ideas are complementary. Ideas from Lean come from manufacturing, so applying them to software development requires care and understanding of how software is different, and why. More importantly however, the conflict goes against the spirit of Agile, and does not do the software development community any good. It drives a wedge and a barrier where there should be none. It causes proponents of either camp to ignore or resist good ideas from the other. Primarily this conflict seems driven by personalities in the community who see personal gain and status with attacking another’s work. The idea that in a community of thinkers one can gain reputation by putting down other proven and legitimate ideas is false. Gaining reputation in a community of thinkers such as software development requires that you show leadership in new ideas AND in integrating new ideas with proven ideas already in place. The battle of Scrum vs. Kanban reminds me of kangaroo boxing… Richard Attenborough narrates…

  • -

Mirage Scrum

A recent discussion on the Certified Scrum Trainers list regarding Sprint 0 and what was a product prompted me to write a long reply I have posted here. Mirage Scrum is the kind of Scrum that some people practice, where what is being developed looks like something it is not.

This is a true story describing a project that used what I am calling Mirage Scrum. The Scrummaster on this project was a colleague that I advised.

The VP in charge of a large program to integrate a bank’s online products came under the spell of a Boston Consulting MBA consultant with no dev experience and a user experience fixation. Based on the consultant’s advice, a project was setup to create the “vision” of this project, in this case the User Interface for the integrated product portal. The project team was to design the interface using a tool called iRise, that creates clickable UIs, along with typical graphic design tools.

For 3 months this project team worked in creating this User Interface (UI) using Scrum. They had a consultant Scrummaster, a scrumboard, demos, and 2 week sprints. They would demonstrate progress on the UI in the iRise tool. After 3 months they had finished the UI design in the tool, and shown it to all the executives in the bank, who approved, though many wanted to adjust colors, move things around on the screen etc. Everyone has an opinion in UI.

This UI concept was then given to the development organization. Then the fun began.

The iRise tool claimed to generate all the code needed. Well, what it generated was not usable for much of the site, so development decided on PHP. The UI design made many assumptions about the backend that were wrong, so many UI elements had to be reworked. The UI design did not have the correct security interface. Etc.

The executives who saw the completed UI in iRise asked why, 4 months after the final interface demo, was the portal not done?? Development needed to go faster. Marketing projects had been started to market the online portal based in part on seeing the completed UI in iRise. Investments were being made with ad agencies, printers, TV, media and online.

Yet massive data-warehouses had to be developed to integrate data and abstract the portal away from the many disparate systems. Acquisitions that had completely different systems and environments had to be integrated. A very large amount of development remained. This development needed to be rationalized against the UI concept, and tradeoffs, changes, and revisions made. It took a year to get the first version of the portal launched, and multiple releases to complete all of the features needed by each product.

So did the UI project add value? Yes there was value, the same value as there is in a requirements document in a waterfall project. Except that the UI project, with its demos of clickable UIs, created an impression of “done”. Based on what Managers saw in the demos, decisions were made and investments taken. Stakeholders believed the Mirage the UI team had created using Scrum practices. You can imagine the conversations between IT and marketing for the following 12 months. Many expectations to manage, IT seen as slow and always making excuses, money lost on purchased advertising space option contracts, and ad agencies doing multiple revisions to marketing material based on the changes in the system under development.

As an Agile Coach in the organization I consulted the SM and tried to persuade the consultant early in the program to find another way that would create real software. Instead of a UI based on assumptions. The consultant had lots of political support for his approach. He said the Company had lots of “consumer anthropology” that needed to be done that could not wait for real software. Since arrowheads and pottery shards were in short supply, I think he meant showing the UI to users, perhaps doing some usabilty tests? I am still not quite sure.

Can you create a vision with using scrum practices? Absolutely. Are you implementing waterfall in 2 week sprints? Absolutely.

Mirage Scrum. Using Scrum practices to deliver artifacts that create the mirage of completed software.

  • -

Agile 2010 Talks on Large Scale Agile and Coaching Teams

Agile 2010 is only a few weeks away, and it should be a very good conference this year. After the being the Producer of last year’s main stage, I am looking forward to being more involved with doing presentations and participating as both a speaker and a listener. I also really enjoy this conference for the all the great people I spend time with at the talks and in the halls. I hope we will see you in Orlando!

On Tuesday August 10 I will co-presenting with Jurgen deSmet on the Large Scale and Distributed Agile Stage.

Cooking the Product Stew

What is a stew? A stew is made from various ingredients you have around, left overs, new ingredients that go together well, and others that are filling but need lots of sauce. The enterprise software market is made up of products that are a stew of software on many legacy platforms that have evolved over a long period by many hands. Taking an enterprise software product to Agile methods is a challenge. In a uniquely European context, this presentation will draw from the ongoing agile adoption in a multi-location multi-team enterprise product with 460 staff in 4 countries and 5 locations.

On Wednesday August 11 I will be co-presenting with Yves Hanoulle on the Agile Coaching Stage.

What I learned from burning my parents’ house down!

In July 1991 Yves’ parents went on holiday leaving him (19 years old) to guard their house. On August 1 at 19 hour 36 minutes Yves parent’s house burns down. It was Yves fault. And yet it was the best thing that happened to him till 2002. If you want to know why, come to our session. This interactive talk will show you why a crisis is a good thing. We make the link to agile transitions and how coaches can use a crisis. Why do people change? Do we consider their interests when making a change? This talk is based in part on the book “A Sense of Urgency” by change management expert John Kotter.

  • -

Cooking the Product Stew, a Keynote presentation from Agile Eastern Europe 2009

In September this year I gave a keynote presentation with Jurgen De Smet at the first Agile Eastern Europe conference. Our presentation was about a company that over 12 years and many acquisitions has grown a software development environment that is very difficult to manage. With humour and the cooking show format we take a look at the dysfunction and complexity in this environment, how it arose, and discuss approaches we would take to improve the situation. Enjoy!

Robin Dymond and Jurgen De Smet, "Cooking the Product Stew", Agileee conference, 2009 from Alexey Krivitsky on Vimeo.