Tag Archives: failure

  • -

How do we get to the customer?? A startup with a great idea, an identified need, a prototype and…

Tags : 

I was recently cleaning out a storage area and came across one of the prototypes from a startup I co-founded in 1993. It is a prototype battery powered portable digital audio sound processor for the hearing impaired.

Background
High quality audio processor prototype for hearing impaired
At the time my partner and I were working on a joint venture project to pioneer the first generation of all digital hearing aids. The hearing aid project was tough because the enormous constraints of a very small device that will run on very low power continuously for days. In learning about hearing aids we realized that they are not good for listening to music. Hearing aids have a very limited frequency range, so low and high frequencies are not amplified. Listening through hearing aids is sort of like listening to an AM radio, but worse. The limitations of analog technology caused a technical barrier to better quality. The other main limitation was that hearing aid users wore them for long periods so they demanded convenience and small discrete size above all else.

The big idea

We realized that the hearing impaired could hear much more music and have a great listening experience if they had a device that would treat the full spectrum of sound available to their impaired ears. Headphone, Walkmans, and portable CD players were common, so people were willing to carry music devices. We envisioned a portable device that you could take to the concert or use in your home.

Testing our idea
inside the prototype
To be able to test our idea we ran some quick tests using a PC with an audio card. We processed some music with hearing loss algorithms and did A/B testing with a few hearing impaired to see if they notice a difference. We got mildly positive results. However testing in a Lab is quite different than a user using this processing in their own home or wherever they needed it. We also wanted to be able to test different settings and algorithms for compensating hearing loss for music. This meant we had to build a portable prototype that would allow us to program different processing ideas quickly, yet would run for a few hours on a battery. We also needed to do it for the lowest cost in the shortest amount of time, since this was money from our own pockets and time from our evenings and weekends.

Building the prototype

Motorola 56002 Digital Signal Processor Motorola 56002 DSP Evaluation board
PC JTAG interface audio inputs and outputs User interface
We built the prototype using a combination of off the shelf components and custom circuits. The first consideration was could we power the device circuitry using a battery? We discovered we could use a large rechargeable cell phone battery and a switching power supply component to get the power we needed. Around that time Motorola released an evaluation board for their 56002 Digital Signal Processor (DSP). I had been programming this DSP off an on for 4 years and it was excellent for audio processing, so we had our main logic board and processor. Most evaluation systems need some adjusting to your application, and this was the case for us. In additions to adding battery driven power supply, we needed to increase the system’s memory and add flash memory to store the settings and programs. We also needed to add buttons and LEDs for users to control the settings and volume. We also added a high quality digital audio interface to connect headphones, a microphone and line audio inputs for sources such as a CD player.

Testing with users

A number of hearing impaired users volunteered to test the system for us. We set them up with 4 different programs that they could select and a volume control they could adjust. We then sent them off with the device in a waist pack with the goal of using the device whenever they listened to music or went to a show. We also asked them to compare the quality of music and audio to their hearing aids. In the device we could track what programs and volume levels they were using. After a number of field tests, users reported mild improvements in listening to music, and a preference for certain settings in the device. The algorithms I had developed were working, however I knew we could continue to greatly improve the effectiveness of the hearing compensation processing.

The missing link – distribution

We proceeded with marketing and sales discussions with Audiologists. Audiologists are the Medical clinicians who test people’s hearing and prescribe hearing aids. This was the natural channel for our device, since it would need to be customized for the hearing impairment of the user. In learning about Audiologists’ business we discovered hearing aid manufacturers were heavily incentivizing them to sell hearing aids with high margins, discounts and free trips. A good quality hearing aid cost on the order of $800 to $1000 per device. Because Audiologists were focused on selling hearing aids to improve speech perception, there was little interest in offering our product to the hearing impaired. In doing our market research we found a simple device that offered some of our features for amplifying audio without the ability to do custom hearing compensation. Sales for this device were struggling and Audiologists were lukewarm to the device. Hearing aids were so expensive that most customers would only pay for them. The high margins hearing aids provided Audiologists meant they’d rather sell and fit hearing aids then a specialized device for music and audio.

After more conversations and demonstrations we put the business idea and this prototype on the shelf. We saw that while there was a need in the market and the technical solution to meet that need, the realities of the distribution channel meant the device would likely not succeed in reaching the customers. Trying to build an independent marketing and distribution channel through pro-audio retail channels such as stores was explored, but looked overwhelming and risky. The devices required hearing testing for baseline data, hearing compensation and user validation, a process far too complicated for a retail setting.

Our expensive lesson: If you are doing a Startup and using Lean Startup methods, identifying a need in the market and the solution is not enough. We found a need and a solution, we tested it and proved our solution worked. We discovered that you also need to prove you can get your solution into your customer’s hands. Perhaps now, 20 years later, we could do something similar with a smart phone. The challenge of getting the product properly configured for a user’s specific hearing loss remains.

I am ever grateful to my fellow investors/learners in this experiment:
Dr. Abram Gamer and Don LaFont.
– Robin Dymond.

  • -

How Healthcare.gov could have saved billions of dollars and been delivered in 1/2 the time.

Tags : 

By September 2014 spending on the 15 state health insurance exchanges and healthcare.gov will climb to over $8 Billion dollars*. This huge expenditure for health insurance shopping sites could have been avoided if the federal and state governments had mandated and followed modern software development practices.

onebillionincash

$1 Billon USD. We’ll need eight for healthcare shopping sites.

How did the governments, on something as high profile as healthcare reform, decide to use a risky 40 year old process to manage the delivery of the health insurance exchanges?

Comparisons were made between healthcare.gov and amazon.com, yet the way in which these two websites are developed could not be more different. Healthcare.gov used the phased or waterfall approach with 55 different contractors responsible for different aspects, and no one responsible for delivering finished product. Amazon.com uses Scrum, an Agile approach that emphasizes small cross functional teams who deliver working tested features every 2 weeks.

To understand how a website like healthcare.gov could have been delivered using the Amazon.com approach, I created a short illustrated video. This video demonstrates, in a simple way, how to deliver a site like a health insurance exchange using a fraction of the budget in about half the time. These techniques are very similar to how companies like Spotify, Google, Square, Valve, Salesforce, Amazon and many others manage getting software development done.

Got a few minutes to save billions of dollars on software development?

I hope you like this talk, please subscribe on youtube if you are interested in future videos. If you are looking for in person training for yourself or help for your organization, please contact me:

https://www.innovel.net or http://www.scrumtraining.com

Cheers
Robin Dymond, CST

*Health and Human Services data

*Report by Jay Angoff on Health Exchange enrollment costs per state


  • -

App Store’s Little Shop of User Interface Horrors

Tags : 

I am trying to find a specific kind of iOS application. I am using the App Store in iTunes, on a Macbook Pro with a 27″ external monitor, extended keyboard, and wireless mouse. But it doesn’t really matter. I could be on my ipad, and the experience would be just as frustrating. Apple says that they have 500,000 apps available for iOS. This is a big point of pride for Apple. Well let’s look at all the tools Apple gives iOS users to navigate 500,000 Apps to find what they want.

I would like to find a keyboard utility the can replace the iPad software keyboard with one that has arrow keys for editing and is more accurate. I don’t know if such a utility exists, however the keyboard and text editing capabilities of the iPad are a regular user complaint. I open iTunes and click on iTunes Store and then the app store. See the screen shot below. I am confronted with a giant rotating advertising banner that often features Apple products like ibooks 2 and iTunes U. Scanning the rest of the page I see headings New and Noteworthy, and Quicklinks, this section includes additional lists featuring Apps chosen by Apple.

Apple App Store Front

The Apple App Store Front

On the right there is a “Top Charts” list featuring more Apps that are the top of some metric, also controlled by Apple. Other Navigation headings include the enlightening “What’s Hot” section, featuring another Apple controlled list of Apps. Scrolling down the page are 7 more lists defined by Apple staff.

Futile Browsing
What if you know generally what kind of App you want? Well out of the whole first page of the App Store, there is only one small button that might take you to where you want to go. Under Quicklinks is a dropdown list with broad categories. How long do you think it would take for most users to find this single button with its hidden categories? Well, they probably wouldn’t. I did, because I am writing this article and went over the App Store’s front page very carefully. Clicking on the drop down I am confronted with a long list in very small font, of all the categories. After a couple tries I click on the category Utilities.

Utils

iPad Utilities Landing page. No further categories, more Apple controlled lists. Click image to see UI detail.

The Utilities page loads more Apple lists, with New and What’s Hot again listing Apps that Apple has chosen to feature. This is the end of the category navigation. Only after scrolling down do we find the All Utilities iPad Apps section. The section has 3960 applications, of which we can see the first 24 with listed with “most recently released” at the top.

App Store Ipad Utilities landing page screenshot

App Store Ipad Utilities page, 3960 Apps

To find a keyboard I re-order the list and click on “K-O” since I am looking for a keyboard utility. iTunes returns 180 applications from App iXML to LogVu. I find 2 Apps with the first word Keyboard. After viewing an App I click the Back button, however the list is different and I am disoriented. The 3960 Apps are reordered by release date again and I am back on the first page. After multiple tries I give up on browsing. However there are likely other keyboard utilities that use different names, so I will try another approach.

Searching for Something?
ITunes has a global search feature in the top right of the application. searching “keyboard” in this search utility returns results from movies, music and every other section of the iTunes store. Filtering by Apps, and then again by iPad Apps returns all iPad Apps for “keyboard” a whopping 581 entries, all conveniently sorted by well, actually I cannot determine how they are sorted, and I can’t change the sort. My search for a keyboard utility has returned Apps from almost every category, including Utilities. More specific search terms, such as “keyboard utility” or “keypad” or “arrow keys” returned more specific results, including everything from emoticon Apps to remote controls for PC and Mac applications. Apparently it is easier to turn the iPad into a keyboard for a PC application then it is to create a different keyboard for the iPad itself.

Searching returns long unordered lists with no descriptions


Power Search: Did you Miss it? I did.

In doing the search I stumbled upon the Power Search feature. Power Search allows you to select a more detailed search criteria. Where do you find it? Well that’s a good question as power search appears and disappears from the App Store UI, even while you use it. It also doesn’t seem work, missing some Apps, or returning Apps not in the category selected.

Too little information
As can been seen in the figure above, search results include a picture, the title of the application, the date it was updated and the price. There is literally no way to judge whether an App is worth your time unless you click on it and view the application’s page. Even on this page critical information about the App is hidden from view on the App’s own page, Users are forced to click again to read more than the first 2 lines of the App’s description.

Useful information is hidden from users on the App's own page

Your App is lost in the App Store
App developers should be very concerned about How Apple is controlling access to their applications. By not building in effective browsing and search features, Apple is preventing customers finding your application. This means all the time, effort, and money spent on your App is wasted, since unless the marketers running the App Store choose to promote your App in the Apple controlled lists, potential customers can’t find your product.

A lesson from Amazon and Musician’s Friend
Apple customers and iOS Developers both lose with Apple’s App Store User Interface. Navigating the App Store is painful, slow, and ineffective. It is clear that the App Store in iTunes was never designed to handle an inventory of 500,000 Apps. The App Store and the other storefronts in iTunes need a major investment to bring them up to the ecommerce standards of today. Instead of providing effective tools that allow users to find the Apps they need, Apple is pushing dozens of lists, in the hope that Apple taste makers know what Apps their customers want. This is bad for the App market, bad for App developers, and very bad for App Store customers. Apple needs to take a lesson from ecommerce sites such as Musician’s Friend and Amazon. Those sites handle as many or more products then the App Store and they provide the tools users need to find the products they want. App store users, whether on the PC, iPad or iPhone need to be able to browse and search effectively to find the Apps they need. Today the App Store is a little shop of horrors to use, devouring both customer’s and developer’s time and money. I still have not found that keyboard utility to replace Apple’s iPad keyboard, so I will continue to reach for the laptop and leave the emails for the Android powered Motorola Droid Pro World phone with its real keyboard.


  • -

Scrum adoption and comments on Martin Fowler’s Scrum post

Tags : 

Martin Fowler recently posted the blog entry “Flaccid Scrum”. Martin makes some good points on Scrum and its adoption without the corresponding agile engineering practices. The observations from his colleagues are common, as teams that are new to Scrum and Agile techniques are learning a new way of working. Scrum implementations lend themselves to adoption of Agile engineering practices. The messy code problem is not a result of Scrum, it is Scrum that has brought visibility to low quality code. Having to put code into production early and often forces teams to recognize defects that otherwise would remain hidden for months in a waterfall process.

Semantic diffusion – Martin’s term for the observation that definitions become diffused over time is an interesting idea. In doing Agile adoption at a major Financial organization we called this “death by a thousand copies.” Someone would have a practices based definition of what Scrum was and then share it with other people, with no references back to the underlying principles. They would modify practices to suit their environment, and those practices would become the defacto “Scrum” for that group. Why does does this occur? We found it was because people did not understand the basic fundamental principles of “why” certain practices are put in place. In the market we see many “ScrumBut” implementations of Scrum, where the basic principles and practices of Scrum are are known and understood but not followed. This phenomena is not just a Scrum problem, Object Oriented development and adoption of Lean principles in manufacturing (outside of Toyota) have also been poorly implemented at various organizations. Each implementation however poorly done, gave some benefit to the organization, often a short term benefit. For example car manufacturers like GM learned how to lower inventory and remove waste from their manufacturing plants.

The time tested way to deal with semantic diffusion, death by a thousand copies, and poor quality code, is to implement two foundational values: respect for people and continuous improvement. Respect for people in Scrum is articulated in ideas such as self organizing teams, pull, and commitment based planning. In Lean it is exemplified in the authority that any worker can and should “stop the line” to address defects and implement solutions that eliminate the root cause of the defect. There are many other examples of respect for people. At Toyota employees are expected to continually improve the process, and if it eliminates the job they were doing the company will find another place they can continue to make improvements. In August 2008 Toyota closed two truck plants in the US but did not do any layoffs, instead putting 4500 employees into a massive training program. This security empowers people to implement improvements knowing they do not need to “protect” their current job on the assembly line. Compare this to the Financial organization that employs talented software engineers to implement repetitive tedious manual production release processes instead of empowering these people to build standardized automated processes. Those same engineers have no interest nor are they asked to implement these automated systems because it would threaten the existence of their jobs and their manager’s jobs.

Continuous improvement, also called Kaizen by the Japanese, is the daily work of finding and implementing improvements to the process. In software the process is how to go from an idea to working testing software in production. Those improvements should result in ever faster translation of ideas into software with less effort and less cost. In North America we are obsessed with Goals and Objectives. However what happens if our goal is to never be satisfied with: the current process of delivering value, the quality, and the cost? How do you achieve perfection in the delivery of software for your customers? How do you do that in the shortest cycle possible?

Scrum does not tell software developers to use test driven development, continuous integration, pairing, or automated testing. Scrum was created that way on purpose. Once you start using Scrum, implementing these practices should be part of the continuous improvement cycle. Learn Scrum, learn XP, learn Lean, learn Theory of constraints, and most importantly, learn what your customers need and strive to fill those needs using these great ideas.


  • 7

A Scrum project that failed.

Tags : 

It is hard to talk about failure. However when you work with many projects and many teams, the odds are good you will experience failures. Last year I had such a learning experience, and one year later it is far enough away that I feel I can talk publicly about it. This project started with the best of intentions. The business case was developed and sponsored by two directors who saw an opportunity to improve a data gathering process that was used to define and market products. The current process involved manual entry of data into a 100 row by 200 column spreadsheet, multiple sources of data, many approvals, and finally re-keying of this data into a custom JAVA application to feed downstream systems. The goal was to replace all of this with a COTS application that would allow re-use of data, the analogy was a “catalog.” A vendor with some presence in a related area was bringing new enhancements that was going to allow this capability. We began a Scrum project to implement this new COTS application to replace the manual entry, spreadsheet and custom JAVA application. A knowledgeable Product Owner with deep experience in the current process was recruited, a team with experience in that part of the business was brought together, and a senior Agile coach who knew the organization and had worked with many teams and some of the team members came onboard (me). In addition a big 3 consulting company with experience with the product was hired to provide team members and guidance. How could a project with all of these things going for it get off the rails? Let’s take it apart and see what we can learn:

Iteration 1: Got initial COTS application installed, knowledgeable PO from business recruited, and team put in place.

Iteration 2: Learned about development/configuration of COTS tool, built out product backlog, and investigated incremental replacement strategies for current system. Product Owner begins to doubt the capabilities of the tool to deliver the needed capability. Product Owner does not see a reason to change a process that they constructed and has been used as is for 3 years. Big 3 consulting firm puts 4 people on the team though 2 members have little experience with the tool.

Iteration 3: Team delivers first functionality in COTS dev environment and pilots it with users. COTS tool still lacks key features required. Feedback is universally negative from business users. Sponsors make a decision to not show the app to customers for next 6 months in order to not create a bad impression with customers. The Product Owner is felt to lack vision and is soon replaced by a Director. An IT Lead with little technical skill and a strong controlling personality is put on the team to “shake things up.”

Iteration 4: Vendor onsite promising features “soon” and working with team to define how system could work. Internal IT controls require a work orders for every db change, causing the team’s pace to crawl, as UI changes in the COTS app generates database changes. Team works around issue by adopting completely local dev setup. IT lead begins attempting to control information flow to team, complains that Scrum is “loosy goosey” and that we need a schedule to drive towards.

Iteration 5/6: Scrum Master and Team members escalate continuing conflict with IT lead, action taken to reduce his involvement. Directors decide to wrap up contract with Big 3 consulting firm. Vendor working daily with team, however key features of COTS package are still not delivered. Business stakeholders who are not engaged in the project but have their “sources” begin influencing the project sponsors and question spending. Process improvement initiative that has been on the back burner for 1 year starts to gain momentum on business side.

Iteration 7: Vendor delivers first version of features with numerous limitations. Directors now openly discussing shutting the project down.

Iteration 8/9/10: About 1/2 the staff roll off the project, smaller technical team continues to try make progress with tool.
Project begins winding up.

Looking at this project we can see that while the Directors did many things to try ensure success, including using Scrum, hiring a big 3 consulting firm, putting lots of people on the team, etc. However there are 2 root causes that are pretty clear:

Project Inception:

The project business case was driven by an IT Director and an OPs Director working together. The business that owned the current process (remember this is product definition data that is generated by the business) and that giant spreadsheet was not driving the project and asking for these improvements.

Platform Decision:

The choice of platform was driven by a belief that the infrastructure needed to move to a COTS application vs. a custom application. The choice of the application was based on promised features that had not yet been released in the application. Further, the application was chosen prior to the start of the project and before anyone had tried to use the tool to build the desired capabilities. Furthermore, when the application was used by the team it was found to be very limited, and the business users found it painful to use. This data was rejected by the Directors. Instead of replacing the tool they chose to wait for it to improve.

These two issues in my estimation doomed the project way before any of us knew we would fail. Further when the warning signs started to crop up there was a failure to recognize and act on this data.

If we look at the principles of Lean and Agile we see a number of violations. If the principles had been followed, the project might have been successful, or terminated much earlier, and either outcome would have been better for the business.

1. Know your customer, engage them and do what they need.
The business stakeholders that owned this process were not engaged. Had they been engaged, they could have influenced process changes that would have enabled a more effective automation solution to be developed. When the business was engaged through the product owner, they were unsure and then convinced the tool could not do what they required. They also did not see a need to change the process they had in place.

2. Decide at the Last Responsible moment
The Directors became emotionally invested in the vendors proposed solution long before the solution was available. This meant that other options, such as a custom application were never evaluated. A set based approach where multiple potential solutions were explored would have provided a much higher level of learning and much better data to base the platform decision upon.

3. Working software is the primary measure of progress
When in iteration 3 the team delivered the first capabilities in the tool the customer feedback was hard for the stakeholders to hear. They did not like what they were trying to use. The response was to stop inviting users to try the software. This is clearly denial of the reality that the project was at risk of ever meeting user expectations, and a better response would have been to change the tool or cancel the project and spend the money elsewhere.

Interestingly, while this IT project ended up delivering nothing, a new project was by the business started to implement major process improvements based on Lean. We also used Scrum for that project and the project was a huge success. I will be cover it in a later posting.

Interested to hear your thoughts, have you had similar experiences?

Post Update based on scrumdevelopment list discussion.

No one starts out a project or a mountain climb with the intention to fail. My hope with this blog and ensuing discussion was to show the process of how events unfolded that lead to the failure. I believe it was the process and a series of decisions that lead to the failure. In the case of this project, the average team member’s years of experience was around 8, and key roles were played by experienced staff some with an MBA from the top three schools. About half of the team had recently completed agile projects that were successful, one being changes to the very same JAVA application that the new system would replace. I would start another project with the same people in that same business area today.

A bit about the COTS application and why it was picked. The COTs application was used by other customers for custom forms based workflows for operational (standardized) projects. The platform supported a high level of configuration for this type of work. One of the goals of the project was to provide a system that would allow business users more control over their work, where they could use configuration to implement automation instead of requiring IT staff to code. A configurable process tool would replace Excel and manual processes that made up the front end. So the COTS application was one of a few from a reputable vendor, had some capabilities in production and a customer base. A key capability it did not have was the (at the risk of oversimplifying again) ability to replace that huge spreadsheet with an automated reusable system – the “catalog” I mentioned. That part was slide ware, though it was heavily sold by the vendor.

The organizational structure played a role as well. The silos of business (generate ideas), operations (run the engine), and IT (build new systems) did not work together. The “pain” and cost being addressed was experienced by operations, who were used to requesting systems from IT. The business silo did not understand the cost of their manual processes downstream. This project was trying to bring process improvement, data re-use and automation in this critical part of the business. However the business users, many of them analysts, were comfortable making updates to an Excel file and sending it on, for someone else to worry about things like reconciliation against other data/constraints/products. While data re-use was happening, it was invisible to anyone but the person copying and pasting. Bringing an automation solution might have meant more work for those analysts, but a major time /effort savings down the line. When we started to see the limitations of the system for business users, the time savings for the whole production line was one justification for continuing.