In September 2010 I gave a keynote talk at Agile Eastern Europe called “Dude, where’s my backlog?” The talk covers the common pitfalls seen with Product Owners and the Product Backlog. The talk covers the people that bring a product backlog into existence and the processes they can use to maintain it. This includes using ideas from Kanban for personal process and modifications to Scrum for more frequent involvement of the team in maintaining the backlog.
I was a technology Director in a large web design company 6 years ago, and they failed to adopt Scrum. There were numerous management dysfunctions, however the Creative managers were the most resistant. Primarily it was a case of not wanting reality to hinder the pretty designs they were making in Photoshop or Illustrator with the reality of building enterprise software. Of course there were huge issues when these artifacts met the reality of enterprise architectures on the development floor. This dysfunction resulted in missed deadlines, high stress, low code quality, and 40% annual turnover in IT staff (and pretty graphics). Being at this company spurred my interest in UX and Agile, and I have since found basic solutions that work when people want to do Agile. To date I would say that of all the disciplines in creating software and especially web sites, UX people are the slowest and most resistant to adopting Agile principles and practices.
Here is the way I work with UI Designers and Information Architects who want to work Agile:
1. Product backlog and its priorities drives all the work. So we work from business priorities not UI priorities.
2. Sketch the highest level level UI for the site. No drill downs into buy flows, just top level.
3. Look at the backlog and start thinking about what the team will need for UI 1/2 an iteration ahead.
4. Make paper prototypes (sketches, paper menus, paper buttons) to support upcoming user stories for next sprint. Don’t embellish with new features UX thinks would be cool. KISS. Include validation and other acceptance criteria for fields. Reference style guide as required. Reference page templates as required. I call this a 1 page design spec.
5. Review prototypes with PO, QA and lead dev a few days before sprint planning, ensure they think it is good enough and it is testable. Keep notes regarding potential trade offs.
5a. Check in design docs, version them.
6. Participate in sprint planing with rest of team.
7. Work with team on implementation, clarify details of design (dimensions, locations, behavior, etc.). Work with QA on test plans for new UI. Help build UI (HMTL/CSS/PHP/etc) in dev IDE, pair with dev as they wire it up. Pair with QA to run test cases and implement automated UI testing (Ruby/WATIR, etc).
8. repeat for every sprint.
To recap the UI designer starts 1/2 sprint ahead of the team, helps the Product Owner with UI issues, and creates very lightweight and flexible prototypes as input to sprint planning. The rest of the work is done within the sprint. I have found this is “just enough” lead time to have thought through UI issues without creating solutions in a vacuum away from the team.
Additional tactics to include:
Create a style guide for the team and have them read it, listen to their feedback and improve it.
Create templates: every page is not a unique creation. Refer to those templates in prototypes and design spec.
Break dependencies between UI/layout and business logic at beginning of the iteration. Agree on data/fields/controls at beginning so business logic can be rendered to a very simple layout free page.
Test your paper prototypes with users by asking them to do certain operations and then observing the resulting actions. Don’t worry about big usability testing at end, do it often and informally with people who are easy to access and know enough about that business.
AVOID COMPLEX HIGH FIDELITY USER INTERFACE DESIGN TOOLS. They really slow you down and lock in decisions far too early. These decision are often wrong.
The root cause of an unstable UI design is usually a lack of detailed knowledge about the customer requirements and technical limitations. The product backlog and its user stories and acceptance criteria are probably not in very good shape. To cover these gaps the UI person has to go find out a lot of detail to create the design, which slows them down and results in numerous design changes during the sprint. The UI designer is likely having to make up for lack of knowledge or lack of effort on the part of the Product Owner. Fix this root cause by getting the Product Owner to improve the product backlog so the designer and the team have better user stories and acceptance criteria coming into the sprint. User Interface work is simply another form of software requirement, as are software architecture, features, and test plans. All of these activities can be done in a “just in time” and emergent fashion and result in consistent architectures and usable interfaces. Many are doing it today. However it does require people to change their behavior, and that is likely the hardest thing to do.
Over the last 2 years we’ve seen some pretty hard times in economies in Europe and especially the USA. Financial services and lending business were in an state of near collapse. With each day companies were announcing job cuts. Job losses in the US are higher than they have been in 30 years. Now that the economy has stopped shrinking, and markets are starting to improve, companies are facing a much more difficult business situation. Stock prices and profits have recovered in some sectors, however most companies face slow growth, and systemic changes in their marketplaces. The US consumer has lost a substantial amount of net worth. That group is not coming roaring back to consume like they have in the past. Many of the businesses that cut staff will not be hiring them back, either because they don’t need the capacity, are doing something different, or are out of business.
Many State and local governments are facing heavy shortfalls in revenue. One important reason for declining tax revenue is property taxes. Property taxes are hard to collect when the property is in foreclosure. Commercial properties are also strapped as their clients break leases or only make limited payments. These losses often result in commercial property owners going out of business. The commercial property failures significantly lag consumer mortgage foreclosures, since commercial properties are not affected until their clients start going out of business. Booming government deficits, high unemployment, slow growth, and increased competition in a global marketplace.
As bad as the economy is now, I am sure it is not as bad as Japan’s collapsed economy just after World War Two. That is when Toyota decided to enter the automotive business. 60 years later Toyota leads the industry by most benchmarks, and are more profitable then the next 4 automotive companies combined. During this recession GM and Chrysler collapsed, and all three big auto companies required billions of taxpayer dollars just to survive as much smaller companies. At the same time Toyota took thousands of idled workers and launched retraining programs so they would be ready for the next generation of manufacturing, new assembly systems, and upcoming products.
The great recession of 2008/9 was bad for airlines, many shrank 10 to 15% and some went bankrupt. During this same recession Southwest Airlines managed to grow by 1%, while paying its pilots more than any other US Carrier. Southwest has been profitable for 36 years straight.
There is a major story that is being completely overlooked by the American Media. That story is how some companies have found a way to gain not one or two years of competitive advantage, but 10-20 years competitive advantage. How some companies consistently innovate and thrive, while others consistently fail to meet the expectations of their customers, employees, investors, and the public. These under-performing companies may have survived the recession, however unlike GM and the banks, there is no bailout coming from cash strapped governments to save them.
What do successful companies like Toyota and Southwest airlines do differently? Can they provide a road map for other organizations? What stopped GM from changing and how to recognize those patterns in other companies?
The New York Times and Wall Street Journal are missing the boat for their readers. There is an important story that European and American business leaders are largely clueless on. I know because of the conversations I have with their VPs, managers and staff. The executives running those companies have a fiduciary duty to get up to speed and start to redesign their organizations so that their companies stop surviving and start thriving.
Halloween is a few weeks away, and the strange world of the dead, undead, partially dead, or dead fashion is inhabiting the shopping malls. Over the last two years I’ve watched another kind of possession has happened in business thinking. Once the darling of Motorola, GE, Bank of America and others Six Sigma has been falling out of favor. Six Sigma’s DMAIC model, its emphasis on gathering statistical data and focus on elimination of variation has floundered in knowledge work. Knowledge work has at its heart, people. There is no machine stamping parts. In knowledge work, the work being done is continuously varying in size and complexity. The work of problem solving is done in people’s brains, and the effective flow of information in the work system controls errors. All of the variation in these complex adaptive systems means trying to eliminate variation is a zombie’s errand. Lean and the Toyota Production System are principles based. Many of the ideas in Lean can be applied to Knowledge Work. Ideas like Respect for people, Pull, Flow, Visual Management, Value Stream Mapping, Systems Thinking and Continuous improvement all make sense in Knowledge Work. These Lean and TPS ideas can be used with systems that adapt and change based on the variation inherent in the work.
Over the last few years Six Sigma consultants have started to re-brand themselves as Lean Six Sigma consultants. The adoption of Lean onto their business cards is partly a survival strategy. If Six Sigma’s not selling and Lean is, then let’s “do” Lean too. The problem is that Lean and TPS come from a very different perspective on work than Six Sigma. For example Six Sigma’s DMAIC process is:
Define the problem, the voice of the customer, and the project goals, specifically.
Measure key aspects of the current process and collect relevant data.
Analyze the data to investigate and verify cause-and-effect relationships. Determine what the relationships are, and attempt to ensure that all factors have been considered.
Improve or optimize the current process based upon data analysis using techniques such as design of experiments, poka yoke or mistake proofing, and standard work to create a new, future state process. Set up pilot runs to establish process capability.
Control the future state process to ensure that any deviations from target are corrected before they result in defects.
There are some major problems with the DMAIC approach when dealing with knowledge work:
Defining the problem with voice of the customer is great. However what if the customer can’t articulate what they want? Every software customer I have worked with did not have a functional crystal ball. They can often describe at a high level their needs, but they do not yet have enough information. The only certainty is that they won’t have enough information. Neither do developers.
The underlying assumption of Six Sigma is that the process under study is a repeatable process producing the same type of work product and subject to some statistically relevant variance. Measuring aspects of the process gives data to better understand where variation occurs. In Knowledge Work such as software development or marketing campaign creation the work is continuously varying so the measurements measure a complex adaptive system, not a repeatable process.
Analyzing the data gathered will provide some insights. Let’s say that data reveals a problem and a solution is found. Putting an improvement in place will definitely help. We may rarely do that type of work again. What happens when down the line a different problem occurs that we have never seen and do not have a measurement for?
How do we know we have the most correct process just from analyzing work in our current process? In Knowledge work, the work itself often impacts the process.
Six Sigma’s underlying assumptions fail to account for complex adaptive work systems that rely on people to do the problem solving and rely on information flow to do error correction. So why do people apply Six Sigma processes to knowledge work and what happens when they do? More on this later…. time to catch a plane.
Software development has its trends and its innovations. The term Agile came about because people practicing different iterative workstyles decided to come together and develop a brand, Agile, that represents their common principles and values. Agile workstyles include Scrum, eXtreme Programming (XP), Crystal Clear, Feature Driven Development (FDD), and Dynamic Systems Development Method DSDM. Then people like Mary and Tom Poppendeick recognized that Agile software development could benefit from the ideas of the Toyota Production System, also called Lean for western implementations. Now some of the proponents of Lean, many of them from the Agile community, have been attacking Agile methods like Scrum, claiming that Lean is “better.” To me this doesn’t make sense. Agile and Lean ideas are complementary. Ideas from Lean come from manufacturing, so applying them to software development requires care and understanding of how software is different, and why. More importantly however, the conflict goes against the spirit of Agile, and does not do the software development community any good. It drives a wedge and a barrier where there should be none. It causes proponents of either camp to ignore or resist good ideas from the other. Primarily this conflict seems driven by personalities in the community who see personal gain and status with attacking another’s work. The idea that in a community of thinkers one can gain reputation by putting down other proven and legitimate ideas is false. Gaining reputation in a community of thinkers such as software development requires that you show leadership in new ideas AND in integrating new ideas with proven ideas already in place. The battle of Scrum vs. Kanban reminds me of kangaroo boxing… Richard Attenborough narrates…
Jim Womack recently published a brilliant eletter on lean.org. In the letter Jim discusses the many challenges GM faces coming out of bankruptcy, and the challenges Toyota faces from its rapid expansion. The eletter is great but there are some choice quotes I would like to publish here:
On the message from GM’s Bankruptcy:
“Yesterday also marked an end of the lean narrative that has been unfolding for thirty years, ever since GM first began to decline in the recession of 1979. David (in fact a team of Davids) finally felled Goliath just as Goliath was finally paying attention to the lean message.”
On GM’s management:
“It is easy to blame GM’s recent management for its troubles. But the senior GM managers I have known – almost all of whom had strong finance backgrounds –were remarkably competent at running the company in the financially oriented, manage-by-results way that had produced success for generations. So the problem is not the individual competence of managers but GM’s irrelevant conception of what management needs to do.”
On what is next for Lean:
“We in the Lean Community find ourselves in the odd position of winning a battle of ideas without actually getting most believers to fully practice their new convictions. And we have as our ideal organization a company that is experiencing significant management and revenue challenges despite “winning” the great contest between modern management and lean management.
… Lean ideas are spreading rapidly to new fields, from the beleaguered financial industry to healthcare to government services. Yet we have not fully defined what lean means in these areas, much less how to implement and sustain it. So the dramatic events of recent weeks are not a time for self-congratulation. Instead, they are a time for modesty and self-reflection – hansei, if you will – as we all struggle with the economic crisis while trying to re-define our own purpose as a Lean Community for the new era ahead.”
Martin Fowler recently posted the blog entry “Flaccid Scrum”. Martin makes some good points on Scrum and its adoption without the corresponding agile engineering practices. The observations from his colleagues are common, as teams that are new to Scrum and Agile techniques are learning a new way of working. Scrum implementations lend themselves to adoption of Agile engineering practices. The messy code problem is not a result of Scrum, it is Scrum that has brought visibility to low quality code. Having to put code into production early and often forces teams to recognize defects that otherwise would remain hidden for months in a waterfall process.
Semantic diffusion – Martin’s term for the observation that definitions become diffused over time is an interesting idea. In doing Agile adoption at a major Financial organization we called this “death by a thousand copies.” Someone would have a practices based definition of what Scrum was and then share it with other people, with no references back to the underlying principles. They would modify practices to suit their environment, and those practices would become the defacto “Scrum” for that group. Why does does this occur? We found it was because people did not understand the basic fundamental principles of “why” certain practices are put in place. In the market we see many “ScrumBut” implementations of Scrum, where the basic principles and practices of Scrum are are known and understood but not followed. This phenomena is not just a Scrum problem, Object Oriented development and adoption of Lean principles in manufacturing (outside of Toyota) have also been poorly implemented at various organizations. Each implementation however poorly done, gave some benefit to the organization, often a short term benefit. For example car manufacturers like GM learned how to lower inventory and remove waste from their manufacturing plants.
The time tested way to deal with semantic diffusion, death by a thousand copies, and poor quality code, is to implement two foundational values: respect for people and continuous improvement. Respect for people in Scrum is articulated in ideas such as self organizing teams, pull, and commitment based planning. In Lean it is exemplified in the authority that any worker can and should “stop the line” to address defects and implement solutions that eliminate the root cause of the defect. There are many other examples of respect for people. At Toyota employees are expected to continually improve the process, and if it eliminates the job they were doing the company will find another place they can continue to make improvements. In August 2008 Toyota closed two truck plants in the US but did not do any layoffs, instead putting 4500 employees into a massive training program. This security empowers people to implement improvements knowing they do not need to “protect” their current job on the assembly line. Compare this to the Financial organization that employs talented software engineers to implement repetitive tedious manual production release processes instead of empowering these people to build standardized automated processes. Those same engineers have no interest nor are they asked to implement these automated systems because it would threaten the existence of their jobs and their manager’s jobs.
Continuous improvement, also called Kaizen by the Japanese, is the daily work of finding and implementing improvements to the process. In software the process is how to go from an idea to working testing software in production. Those improvements should result in ever faster translation of ideas into software with less effort and less cost. In North America we are obsessed with Goals and Objectives. However what happens if our goal is to never be satisfied with: the current process of delivering value, the quality, and the cost? How do you achieve perfection in the delivery of software for your customers? How do you do that in the shortest cycle possible?
Scrum does not tell software developers to use test driven development, continuous integration, pairing, or automated testing. Scrum was created that way on purpose. Once you start using Scrum, implementing these practices should be part of the continuous improvement cycle. Learn Scrum, learn XP, learn Lean, learn Theory of constraints, and most importantly, learn what your customers need and strive to fill those needs using these great ideas.
Kiev Ukraine: After 5 days in Kiev, I learned much about the state of Agile in Ukraine, and the state of business in Ukraine’s capital. My first visit to the Ukraine held many surprises. The downtown area of the capital was interesting, attractive, an similar to other European cities. The architecture combined Soviet style massive design, buildings with intricate architecture similar to classic European buildings, and more modern apartments and offices. Beyond the buildings to the street, I expected to see clear signs of a depressed economic region, like closed shops, bad roads, and inexpensive transportation. However this is the capital of a 45 million strong nation, and I was surprised. Construction cranes were everywhere as new offices and business centers were under development. More surprisingly, it seems the Lada has been replaced by the Porsche Cayenne, Bentley, and High end Mercedes as the cars of choice in Kiev. A Bugatti Veyron? Yes. I saw a higher concentration of high end cars on the bumpy streets in Kiev then I have seen in London, Brussels, or Paris. I quizzed my hosts about this, and heard a combination of embarrassment and frustration. It seems for politicians and bureaucrats of the Orange revolution, Kiev is open for business that will fund their new, rich lifestyle. This was set against the backdrop of the crisis in the world financial markets and massive spending of taxpayer’s money for disastrous portfolios of high risk mortgages and credit insurance swaps. As someone who helps teams and businesses better compete legitimately, seeing the corruption implied was disheartening.
Meeting the people in the Agile community in Kiev and at the Certified Scrum Master class reminds me of why this is a great profession. The Certified Scrum Master class was a mix of software developers, managers, QA, business analysts and even a couple account managers. While half of the students had been doing Scrum, we also had people who were completely new to Scrum and Agile. We generally have smart people in our profession, and this class was no exception. While the class was in English, the exercises were in Russian. My Certified Scrum Master class covers a ton of material on Scrum, Agile principles, roles of Scrum, and how to establish Scrum with a team. The class asked good questions, and were sometimes surprised by the results of the many exercises we did. With this class I dropped the “59 minute Scrum” in favor of the “Save Earth” and “Ball Points” exercises from CST Boris Gloger. The exercises are great ways to show how we can fail or succeed based on our own personal actions, and they also put students in a different frame of mind, loosening their preconceptions from past experiences.
My main challenge with English as a second language classes is usually pacing. I choose words more carefully and I go slower to ensure understanding. With a packed agenda this means something needs to drop, but, like an over eager development team, I sometimes over commit, trying to squeeze in too much material. The rise in knowledge of the students and the deeper experience of the trainers may lead to a 3 day course. Some trainers such as Mike Cohn and Mishkin Berteig have already adopted a longer format, with Mike offering 2 days of training on requirements and planning around the 2 day CSM class.
The Agile Ukraine gathering 6 on Saturday 10/12/08 was a great opportunity to meet more of the Agile community, as over 150 people packed a large conference room for the day’s events. There were numerous talks from Ukraine’s Agile advocates and thought leaders. Alexey Krivitski is the organizer, and single handedly put together this event for the community. I encourage more of Ukraine’s Agilists to lend a hand and take a leading role with Alexey to create sustainable group of organizers of Kiev’s Agile community. Topics at Agile Ukraine gathering included Introduction to Agile, How we do Scrum, Agile Metrics, mixing Agile and RUP, and Agile and business. I had the opportunity to provide the Keynote presentation on Lean, value stream mapping, and how to apply lean principles to Scrum and Agile projects.
I covered the history of the Toyota Production System, and how speed delivers a competitive advantage to businesses. Most of the IT and software teams in the Ukraine are engaged in offshore projects. Lean can provide a great way to highlight what slows their offshore teams down, and gives them a way to articulate this to both management and clients. The culture of eastern Europeans has an advantage over Indian teams when using Agile methods. Transparency means teams need to have courage and be honest with their customers. This means saying no, managing expectations, and building trust. Teams that always say yes, don’t manage customer expectations, and then hide when it becomes clear there is no way the team will meet the commitment is common in offshore teams. The culture of eastern Europe is not socially hierarchical and there is a tolerance for open discussion and debate within groups. This means teams in eastern Europe are biased to Agile methods when compared to teams from hierarchical cultures like India’s caste system. I don’t think Ukraine outsourcing poses any threat to the Indian outsourcing industry, however Agile is a growing market where Ukraine outsourcing can have a competitive advantage, provided that they can demonstrate and market this advantage. I will update later with photos from the events.
– Robin Dymond.
1. I would like to understand Agile methods, where can I go and what should I read?
I recommend reading the first 165 pages of Manager’s guide to Agile and Iterative development. This book provides an understanding of the various flavors of Agile, but also an understanding of why these methods are more effective at delivering projects.
A classic overview on Agile is Martin Fowler’s The New Methodology. Fowler is an expert on software development practices and patterns.
The most popular Agile method is Scrum, it is a project management framework that can be used in both IT and non-IT projects. I recommend taking an Agile Project Management course or the Certified Scrum Master course, it will provide an understanding of how to implement Scrum on a project, and the differences between Agile and waterfall methods. Our web site Scrum Training lists public classes we are offering across the country.
2. I understand that Agile is a very different way of working from our current waterfall environment. What are the training options for my company?
Training is key to success with Agile and Lean work styles. We recommend to all our clients a training plan that includes training for all levels in the organization.
Team members: Introduction to Lean and Agile training (1-2 days) for new teams or team members who will be working on an Agile project.
PMs and Team Leads: Agile Project Management or Certified Scrum Master training for Agile project leaders and key stakeholders
Customers, Sponsors: Product Owner Training for customers, sponsors and key business stakeholders
Directors, Functional managers and Program managers who will manage multiple Agile projects: Lean and Agile Management training.
For staff in process definition roles we also recommend:
Process engineers and managers building Lean processes: Introduction to Lean training.
Process engineers and managers leading process improvement projects: Lean Belt Certification
This is baseline training. In addition some excellent training courses that drive improvement in productivity are:
The Secrets of Agile teamwork
Agile User Stories, Estimation, and Planning
Kaizen and KaiKaKu – Lean event facilitation
Innovel can provide all of these training services through our senior training/consulting staff and relationships with experts in the field.
3. How should Agile methods could be prescribed for new projects in my organization that want to go Agile? What projects should we start with?
There is no quick answer for this topic, it is best served by a management consulting engagement where we can work through the unique issues in your corporate environment and develop a strategy that best works for your business. Innovel’s staff have helped a Fortune 200 financial services company transition over 50% of a $1 Billion IT portfolio to Lean and Agile. This includes very complex projects with multiple teams on shore and off shore, to defect and enhancement projects that with 1 team serviced customer changes on multiple systems. Other considerations include how can we maintain regulatory compliance? What about Sarbanes-Oxley and Agile?
4. How about Lean and Six Sigma? How do they fit?
Lean, Six Sigma, and Agile are related. Agile is project centric principles and practices, Lean and Six Sigma are process and organization centric. We recommend adopting Lean at the same time as Agile to avoid local optimization at just the project level and improve organizational effectiveness. We do not recommend Six Sigma in the early stages because it is designed for repeatable manufacturing-like processes as opposed to dynamic product development systems. Six sigma can be used after Lean tools have been applied to production and operational processes.
Please reach out to us if you have any questions or require any additional information on Lean and Agile. We passionately believe in these methods and would be excited to help you and your colleagues discover Lean and Agile.
This 2 day course covers how to use Scrum to manage your Agile projects. Students gain an understanding of Agile principles, Scrum practices, and how to manage the Scrum process. Recommended for project managers, technical managers, and technical leads.
Current classes are listed on the Scrum Training site.
This 2 day immersive training is great for teams who are looking to adopt Agile for their project. Team members come away with a solid understanding of how to work day to day on an Agile project, how it changes their role and what they should expect. It covers Lean, Agile, and Scrum, with a focus on the Scrum project framework.
Many State and local governments are 