Why bugs don’t get fixed
I’ve run into more and more people lately who are astounded that software ships with known bugs. I’m frightened that many of these people are software testers and should know better. First, read this “old” (but good) article from Eric Sink. I doubt I have much to add, but I’ll try.
Many bugs aren’t worth fixing. “What kind of tester are you”, I can hear you shout, “Testers are the champions of quality for the customer!” I’ll repeat myself again (redundantly if I need to …) Many bugs aren’t worth fixing. I’ll tell you why. To fix most bugs, you need to change code. Changing code requires both resources (time), and it introduces risk. It sucks, but it’s true. Sometimes, the risk and investment just aren’t worth it, so bugs don’t get fixed.
The decision to fix or not to fix isn’t (or shouldn’t be) entirely hunch based. I like using the concept of user pain to help make this decision. There are 3 key factors I consider to determine user pain. These are:
1. Severity – what’s the impact of the bug – does it crash the program? Does the customer lose data? Or is it less severe? Is there an easy workaround? Is it just a cosmetic issue?
2. Frequency – how often will users hit this issue? Is it part of the main flow of the program, or is the issue hidden in an obscure feature. Minor issues in mainline scenarios may need to be fixed, but ugly stuff in an obscure feature may slide.
3. Customers Impacted – if you’ve done your work up front, you have an idea of who your customers are, and an idea of how many users are in (or how many you would like to be in) each of your customer segments. From there, you need to determine if the issue will be hit by every user, or just a subset. If you have the ability to track how customers are using your product you can get more accurate data here.
From here, make up a formula. Assign a value scale to each of the above and apply some math – you can do straight addition, multiplication, or add weights based on your application and market. For our purposes, let’s just add and use a 10 pt scale for each bug :}.
Bug #1, for example, is a crashing bug (10pts) in a mainline scenario (10pts) impacting 80% of the customer segment (8pts). At 28pts on the user pain scale, I bet we’re going to fix this one.
Bug #2 is an alignment issue (2pts) in secondary window (2pts) in an area used by a few “legacy” users (2pts). At 6 pts, this is a likely candidate to not get fixed.
Unfortunately, they’re not all that easy. Bug #3 is a data loss bug (10pts). It occurs in one of the main parts of the application, but only under certain circumstances (5pts) (btw – numbers are completely made up and subjective). Customer research shows that it’s hardly ever used (2pts). At 17 pts, this one could go either way. On one hand, it’s probably not worth the investment to fix. As long as the issue is understood, and there are no blind spots, leaving the bug in place is probably the right thing to do.
On the other hand, you have to weigh this with the rest of the bugs in the system. The Broken Window theory applies here – if there are too many of these medium threshold bugs in the app, quality (or at the very least, the perception of quality) will suffer. You need to consider every bug in the system in the context of the rest of the (known) bugs in the system and use this knowledge to figure out where the line is between what gets fixed and what doesn’t get fixed.
It sucks that the industry ships software with known bugs – but given the development tools and languages we have today, there isn’t a sensible alternative.
Edit:
As this sits in my head, I think I've missed a fourth factor in the forumla: Ship Date. The proximity of ship date plays into the fix/don't fix decison as much as the above. I'm not sure, however, whether it's a fourth factor in the math, or if the threshold of what "value" of user pain turns into a bug fix as ship dates approach.
作者：Alan Page， 微软卓越测试工程总监，How We Test Software at Microsoft
翻译：卢玥俪、陆梦嫣、汪宏
近来我遇到越来越多的人对我们会发布还有bug的产品大为惊讶。而让我大吃一惊的是，这些人中还有许多是软件测试人员，我本以为他们应该对此早已经有所了解。建议大家先阅读Eric Sink较早写的（但是很棒的）文章。不知道我还能对此话题有多少贡献，但我想试试。
许多bug并不值得去修复。“你这也算是测试人员吗？”，你肯定会冲我大叫，“测试人员是产品质量的捍卫者。”我可以再重复一次（如果需要的话）许多bug并不值得去修复。“让我来告诉你原因。在大多数情况下，修复bug就必须要修改代码。而修改代码需要投入资源（时间）并会引入风险。这真是很糟糕，但这却是事实。有时，如果风险和投入远超过修复bug的价值，因此我们就不会被修复这些bug。
我们决定是否修复一个bug并不是，也不应该是靠“感觉”。我喜欢用“用户痛苦”的概念来帮助我做决定。我会用三个关键因素来考虑并确定“用户痛苦”：
1、严重性 —— 这个bug将产生什么影响 —— 它会让整个程序崩溃吗？它会导致用户的信息丢失吗？或者并不是那么严重？有更简单的解决方法吗？还是它仅仅是个无关紧要的问题？
2、频繁性 —— 用户碰到这个问题的频率高吗？它是程序主要工作流程中的一部分？还是隐藏在一个并不常用的功能中？在最常用的那部分程序中存在的小问题很可能是需要修复的，而一些不常用到的那部分程序中存在的大问题，也许我们会放在一边。
3、对客户的影响 ——如果你之前准备工作做得好，你应该已经知道你的客户是谁，你的每个客户群中会有多少（或者是你希望有多少）用户。这样你就需要判断，这个问题将会影响到每位用户一，还是仅仅一部分人。如果你能追踪出客户如何使用你的产品，你就能得到更准确的数据。
以上3点因素就构成了一个公式。给上面的每一个因素都分配一个数值范围，并且用一些计算 —— 你可以直接使用加法、乘法或是基于你的应用程序以及市场因素加上权值。打个比方，我们只需要执行加法并且对每个bug赋予10分的数值范围。
Bug #1：比如它是一个会让程序崩溃的bug（10分），它存在于程序的主要部分（10分），它影响了80%的客户（8分），因此这个bug的”用户痛苦“量值为28分，我们打赌我们肯定会修复它。
Bug #2：它仅仅是一个关于排列的bug（2分），它出现在二级窗口中（2分），这个bug所在的那部分程序只会在旧版本中被使用到（2分）。因此这个bug的“用户痛苦” 量值为6分，我们很可能不会去修复它了。
遗憾的是，很多情况并不像上面所说的那么简单。Bug #3是一个数据丢失问题（10分），它存在于一个应用程序的某个主要部分中，却只在某些特定的情况下才出错（5分）（顺便提一下，数据是主观编造出的）。客户研究证明它很少会被使用（2分）。因此它的 “用户痛苦”量值为17分，这是一个模棱两可的数据，修与不修都可以。一方面，修复它所需要的投入可能并不值得，只要这个问题能够被理解，并且它没有任何盲点，不再理会这个bug很可能是正确的处理方法。
从另一方面来看，你必须把它和系统中的其他bug进行权衡。我们在这里应用“破窗效应（Broken Window）”—— 如果应用程序中有太多此类中等阈值的bug，产品的质量（或者最起码，从质量的感觉上）一定大受影响。你在考虑系统中每一个bug的时候，还应该结合考虑系统中其他（已知的）bug，并且以此来分析、决定哪些bug是需要被修复的而哪些则不值得被修复。
正式发布的软件中有bug的确是一件十分糟糕的事 —— 但基于我们现有的开发工具和开发语言，我们还没有找到一个更加合理的解决方法。
补充：
写出这篇文章的时候，我想我遗漏了公式中的第四个因素：发布日期。临近发布日期时，这个因素在修复/不修复bug的决定中也起了关键作用。然而我并不确定它是否是第四个因素，也无法确定在临近发布时期时，修复一个bug所需要的 “用户痛苦”量值的阈值是多少。 |
|
陕公网安备：61019002000258 |
隐私政策 |
网站地图