During the Software Development Life Cycle (SDLC) process challenges await at every stage. Imagine the situation: a new email notification saying that another feature is ready for testing. All right, let’s see if everything works correctly.
All acceptance criteria: green.
Design implementation: green.
All other notes or implementation details added during refinement meeting: green.
Test results: rejected.
Wait, what?! Why would I reject a user story that meets all Definition of Done (DoD) criteria?
The answer is easy – the quality of the feature is still not satisfying. There is still something wrong with the way this particular part of the software works. In other words, there are some hidden acceptance criteria that haven’t been met.
Some of the real-life examples below will help you to work more efficiently during refinement meetings or foresee potential issues with hidden acceptance criteria.
Case #1 – Non-functional requirements not taken into consideration
When describing user story and acceptance criteria, we tend to focus on the business part of the product. We try to answer two main questions: Why the product needs this particular feature (what need will this feature satisfy)? And how it should work in order to bring a valuable solution for the end-users? Unfortunately, very often the ‘how’ question doesn’t consider some technical aspects of the software like its performance, scalability, security or usability. In other words: non-functional requirements.
Example: let’s assume you’re implementing a simple form – a couple of text input fields, one or two select lists and a checkbox. When testing, I would expect all standard key events behaviors, so it should be possible to switch between the elements using tab/shift+tab, it should be possible to select items from the list using key arrows, and it should be possible to submit the form by pressing enter. Otherwise, the form is not usable, a lot of users will be frustrated.
Countermeasure: when designing/analysing/developing any feature, think about at least basic non-functional aspects. Remember that different users will use the product in a different way – some of them will use only the mouse or touchpad, others will try to do as much as possible using only the keyboard.
As both methods are valid, the software should meet standard-behavior expectations. Also, try to work with real-life example data as much as possible – this is the best test data you can get! Its quality and volume can be essential for your product.
Case #2 – Similar elements in the system work differently
I believe that consistency is the key to seamless user experience. When you learn that similar elements of the interface work in the same way (e.g. when all elements that have a specific color are clickable and users can interact with them), using the product becomes intuitive. But as long as it’s relatively easy to catch any inconsistencies on the wireframes or designs, it becomes more tricky when it comes to data we want to compute and present – it is really easy to end up with some confusing discrepancies.
Example: you open a dating service with a search engine and several filters available. One of them is called ‘age’. You select ‘26’ value, the list is refreshed and you see only people that are 26 years old. You open the first profile on the list and in the details there’s information that this person is 25 years old. How come? Well, it’s simple – the database contains the date of birth for each person but the filters in the search engine calculates the age based only on the year while the age displayed in the detailed view takes the month and the day into consideration as well. A small detail that can generate distrust and impact the product’s credibility.
Countermeasure: first of all, if we are dealing with situations like the one described above, both calculations should probably be handled by the same piece of code. But in general, always keep in mind that each software is not only the sum of its features but something greater. All of its elements must work with each other and shouldn’t compromise its integrity. Even before designing or implementing a feature, you should ask yourself how it will impact the already existing parts of the product.
Case #3 – ‘Weird, it works on my machine’
Classic! I bet anyone who works in IT has said or heard that expression. It is tempting to assume that if something works correctly on local environment then it should work the same way everywhere. We often use this ‘shortcut’ in order to save time or when we’re sure of our code. What can go wrong, right? But when you think about the numbers that stand behind the possible hardware/software combos, you might start to lose your confidence.
Example: let’s dive into the world of Android for a moment. A couple of years ago, there was more than 24k unique Android devices out there. Today, it’s probably somewhere around 30k or even more. Add the different versions of Android to this equation and you’ll quickly notice that it’s impossible to guarantee support for every user.
On top of that, some operating systems or browsers might have some additional limitations that can impact your product. For instance, older versions of Internet Explorer browser (I know it is a totally hardcore example but there are a lot of products that need to run without any issues on older IE) have a limit of 4095 CSS rules in one sheet. If you exceeded this limit, you probably wouldn’t recognize your product.
Countermeasure: at the very beginning of the project, make an agreement between the product owner and the development team about which operating systems, which browsers, which devices and which versions should be supported. When you have such clear guidelines, the development and testing should become easier. And you’ll probably hear ‘Weird, it works on my machine’ less often.
Case #4 – Misunderstanding acceptance criteria
Good communication is essential in order to deliver high-quality product that will satisfy the stakeholders and product owner. And vice versa, poor communication can lead to misunderstanding about requirements and cause a lot of rework, especially when working with remote teams.
Another thing is that the knowledge about the product can be transferred only to a part of the team, for example to business analytics. Sometimes it seems very reasonable not to invite the whole team to a meeting. If there are 8 people in the team and the meeting is scheduled to last one hour, it will burn 8 man-hours. It’s a whole day of work for1 person. From the management point of view, this might look like a waste of resources, but very often it’s quite the opposite.
Example: you want to introduce membership plans for the customers who are using your product. They should have the option to choose between monthly (payment is generated each month by the defined period of time but minimum for 12 months) and annual (user pays upfront for the whole year) membership types. In other words – no matter what option they choose, they have to pay for it at least for a year. Each membership includes some paid features that vary in terms of how many times you can use it during a year. One of the developers, who hasn’t discussed this issue with the product owner or anyone else, starts to implement this feature and after reading the description he assumes that if a user chooses the monthly plan, all the paid features and their quantities are refreshed each month. In the case of the annual plan – they are valid for the whole year. That’s faulty logic but since the developer hasn’t discussed the details and his reasoning looked legit, the damage has been done – an incorrectly implemented feature hit the test environment.
Countermeasures: The best remedy for any miscommunication is to have the whole team during any crucial meetings. When you gather all team members in one room when any product related decisions are being made, they will have the first-hand information and you won’t have to distribute the knowledge afterwards. But what’s more important, some additional doubts, concerns, questions might come up during these meetings. So if possible: ask, ask, ask! Better to ask the way than to go astray.
Case #5 – Operations on ‘troublesome’ types of data
As a tester, I have the natural need to try to break the software by feeding it with data that can cause unexpected or unintended behavior. Sometimes the application will crash, sometimes a security hole can be found and sometimes funny glitches can be observed. But sometimes the very nature of the feature or even the whole product is based on data that is troublesome to process.
Example: one of the biggest pains in the you-know-where in the IT world is floating-point arithmetic. You can find it on a daily basis in many applications that allow to process any kind of payments. Let’s assume you want to calculate the total price of a product. Its net price is 99.99 and VAT is set to 8%. Most of the calculators will say that the final price is 107.9892 but normally prices have two decimal places. So the question is: what should be the final price in this case – 107.98 or 107.99? There’s no easy answer as it may vary, for example based on the local law regulations or limitations resulting from 3rd party integrations.
Countermeasures: when it comes to any troublesome data (floating-point arithmetic, dates and time, converting data to different units) you must be more than 100% sure how to handle all possible scenarios before you write the first line of code. The whole team must turn on the ‘what-if’ mindset and try to nitpick as much as possible. Don’t assume anything, common sense might be useless in these kinds of situations. In the end, you probably don’t want to repeat Mars Climate Orbiter mistake.
Case #6 – Integration with an external system
External applications that specialise in specific services are very popular in software development nowadays. When it comes to setting up a payment gateway, onboarding or tutorial materials for new users, in-app chat or mailing solution – you have many options that are available all over the web. You can find paid or open-source products for almost any service you would like to introduce to your application. And the one thing that they have in common is that they are supposedly very easy to integrate with. It’s convenient, right? Why would you want to reinvent the wheel when you have a ready-to-use solution just around the corner? The problem starts when the ‘ready-to-use’ becomes ‘partially-ready-to-use’ or when the setup is more troublesome than you thought initially.
Example: you want to add an internal chat system to your product. You want to have the option to create one-on-one and group conversations between different types of users. And you want to have an option to block any given conversation for any given user so that they cannot send messages anymore in this thread. After looking up the API documentation you find out that it’s possible to disable the chat for a user – there’s a ready endpoint for that. So you think you’re good to go, research done. A couple of sprints later you want to integrate the chat with your application but it turns out that the description of this endpoint was vague – it is possible to block a user but then they cannot access any conversations. And this doesn’t meet your initial requirements, so you have to change the external service, send a change request to service provider or handle this scenario on your side. Probably each of those scenarios will greatly impact the delivery time of the chat feature.
Countermeasures: in my opinion, integration with 3rd party service is by far the most difficult risk to evaluate, unless you have loads of experience in the particular service you want to use. It’s hard to find a perfect fit. If you plan to use an external application in your product, especially when it relates to one of the critical parts of the product, you should do thorough research as soon as possible. And don’t limit yourself to reading the API documentation or contacting the support team. Unfortunately, the documentation can be outdated or even bugged and the ‘support team’ doesn’t equal the ‘technical specialists team’ which means that more than often the marketing guys might provide you with misleading information about the capabilities of their service. Instead, try to establish a connection with the API and test it by successfully completing primary use cases you want to have in your software eventually. And make sure there are no additional non-technical requirements you have to fulfill (e.g. if there’s a time-consuming verification process) or if the service is available in your country – this is especially important when it comes to payment gateway solutions.
Final thoughts
Those are just a few examples of surprises you might experience during the software development life cycle. Some of them might seem very obvious to you, not foreseeing them might look like a rookie mistake. But on the other hand, I’m more than sure that from time to time you are stuck in a moment when all you can do is say: ‘Huh, we haven’t thought about it.’ And you have to remodel the database scheme, redesign the interface, change the logic in the source code or report an issue.
The point is, the sooner it happens, the better – any hidden acceptance criteria discovery will cost less when it occurs during the analysis or design phase. Addressing any unexpected element of the product during later stages may have a direct impact on risk and backlog management. In the end, the domino effect may be triggered, resulting in changes to priorities or scope, or lowering the value of the final product. All those elements impact the quality of the software.
The bottom line is: don’t be just a passive software developer and don’t neglect the analysis. As experience teaches us, the primary source of your knowledge – probably a product owner or domain expert – don’t know what you don’t know in order to deliver the feature that will meet all expectations. In most cases, you have to ask the correct questions to gain the full understanding of the tasks that lie ahead of you. Be vigilant. Be proactive. Share the common responsibility of the product you are taking care of.
