Black Boxes for black box testing

Some time ago I stumbled on James Lyndsay’s Black Box Puzzles. I found them funny and intriguing. Last year at the Autumn Camp (a tester’s gathering in Romania) while attending Alexandra Casapu’s Examine your testing skills I actually saw a couple of real black boxes. I loved them so much that I wanted to build some of my own.
After doing some research and learning a few new thing I managed to build 4 (so far).
Here they are:

black-boxesIf you are interested in building one here are some things to consider

1.   First you need to have a clear idea about what you want to build. What puzzle you want to do. Once that is clear you will need the hardware

2.   Electronics. To make everything work I have used a Arduino nano for all my black boxes. At this stage you need to have a clear idea about what the black box has to do. The nano had 12 digital connection where you can put buttons or what ever you need.

3. The box. Everything has to fit in a box. You have to be very careful to fit everything in, and make sure it is easy to cut the case to fit the buttons or anything else you need for the black box to work .

4. You will also need some tools for making holes in the box, or fitting stuff. I have used a mini bread board to fit the Arduino in the box, a drilling machine to make the holes for buttons and LEDs.

5. Programming the Arduino. It might sound difficult but there is a lot of support on the web. There are plenty of Arduino related forums.  Just ask Google what you need to do. If you can’t figure it out let me know. Perhaps I can help.

6. Powering.  I made mine with a battery included so that you can take them anywhere. You need to search for a battery that does not cut power in case there is a low consumption. Most power packs do this: in case there is a  low consumption, in order to make the battery last longer, they just cut power off after a couple of seconds. The Arduino consumes really little power. For the first black box (3 buttons and a LED), I would say that an average of 12 hours of play lasts with a 2000mAh 5v battery.
Bellow you have some pictures I have taken while making the 2nd one.

Pick an Agile Value or Principle for the day

Before “living to the Agile values and principles” we need first to know and understand them. Here is a fun way to do it: print them as “take away”. Everyone can choose what to take and how to live up to it during the day.

The purpose is to get the people familiarized with the Agile Values and Principles.


Here is the doc if you want to use it.

Learning about processes and tools through story telling.

While hunting for fun stuff I stumbled upon a set of story cubes.  These are cubes with different images on each side. After you throw the cubes you have to make up a story with using the images.  It sounds fun!!! How can I use this? What if I put on the cubes tools, ceremonies, artifacts and other things that I do within my company. I would be able to tell some great stories !!!

After a while I stumbled upon a set of 8 wood cubes for small children. I bought it. I could only use 4 sides of each cube.

cubes animals

I already had some sticky paper that I could print on. Next I made a list of tools, ceremonies, artifacts and other things that I considered useful to know and understand within my company. It turns out that I had more things in mind then there was space. So I added more than one thing on each side with the additional rule that you can choose the item that you want to include in your story.

dices 2 second

I printed them out… cut them and glued them to the cubes. Here is the result:

cub e

It does not look so fancy. I will try it out. See how it goes and if it will be a success then I will find a way to make them look more professional.

The rules of my Cubes Game:

  1. Throw the cubes. If the cube falls with the whole up than that’s it. You will not use anything from that cube.
  2.  From each cube side that has more then one thing,  you must choose only one.
  3. Make up a story with all the topics.


Quality Assurance, Quality Control, Checking and Testing

Within the testing and software development world there seems to be a confusion about Quality Assurance, Quality Control, Checking and Testing. I will show you the differences through my eyes and experience.

Let’s get first a the same feeling about quality:

  • Quality is not an act, it is a habit (Aristotle 384-322 BC)
  • Quality is never an accident; it is always the result of intelligent effort (John Ruskin 1819-1900)
  • Quality means doing it right when no one is looking (Henry Ford 1863 -1947)
  • Cultural resistance to change was one of the biggest problems in reforming quality (Joseph Juran 1904-2008)
  • Quality is everyone’s responsibility and we never have to stop getting better (Edwards Deming 1900-1993)
  • Be a yardstick of quality. Some people aren’t used to an environment where excellence is expected (Steve Jobs 1955-2011)

Quality is difficult to define and explain. It relays to two key elements: perspective and time. Quality lays in the eyes of the beholder. The way you see and perceive it will play a significant role.  Your expectations are also important (take into account that expectations are difficult to state and define) It is also through the perspective of time that you should look at quality. What was a quality attribute a few years ago might be so trivial today that you don’t even take it into account (just think about the difference between an iPod and a Walkman from the ‘90s; would you buy a music player that is at least 10 times larger and heavier with less autonomy than a iPod?).

Since quality plays an important part in the selling and buying of goods and services there are always attempts to quantify it. To determine its components in order to be able to replicate it over and over again. This is the job of the standards. Standards give you a way to assess if your activities comply with a standard set of processes that are considered best practices.

In many situation, when dealing with quality, the focus moves towards the process in detriment of the people that do those activities. It is perceived that a good process is something that can be followed by anyone.  Although this might be true in some industries, I strongly believe that dues to the creativity component this does not apply for software development, especially to testing.

But what is Quality Assurance? Quality Assurance consists of a means of monitoring the software engineering processes and methods used to ensure quality.

In order to ensure quality you need to have a system. This system is usually called Quality Management System. Such a system consists of 3 main parts (written with red letters):


The Quality Assurance part is usually defined starting form a Quality Policy, a document that bears the signature of the CEO. Yes, it should always be the executive that clearly defines what are the processes and standards that have to he followed (this is present in CMMI and ISO standards).  You should see Quality Assurance as “what the company expects you to do in terms of processes and standards”.

Quality Planning is a natural step when a new project or requirements arises. Not all the rules should apply to every situation. Everything is context dependent and the tailoring has to be done at a conscious level. You should not only ignore things that do not apply but you also have to state it and make it transparent to every stakeholder.

Quality Control. Just because it’s define it does not mean it is used. People often disregard processes (and for good reasons). You need to have champions that make sure the process are used and when they are inefficient change and adapt them. Once thing Quality Control should aim for is prevention: if you do something in an unexpected way the thing is done already. All you can do is take a lessons learned and mitigate the damage. Quality Control should create not a checking after is done system but a prevention and alarm system.

Besides checking that processes are respected (understood, used properly) quality control is also responsible for checking the product standards. Checking that the product respects certain standards is one of the reasons why sometimes testing is referred as Quality Control. Quality Control is performed by the use of checks.

Checking can be viewed as “the process of making evaluations by applying algorithmic decision rules to specific observations of a product.” I don’t know how this sounds for you but for me it does not sound like testing. It is not all that I do when I am testing. Testing is much more. It is “exploration, discovery, investigation, and learning”.

 It is true that almost anyone can follow a checklist. You can take all the requirements and make checks for each statement. But Testing is much more. Testing is craft.

I am sure that looking at testing as a craft might be difficult. A craft cannot be steered only by a processes. A craft can be only be learned, in time with patience. Testing also presents a big challenge: practice. How can you practice for the next build? Is there a way? Testing is also based on experience, on heuristics. Heuristics that make you find the unfindable, that guide you towards that scenario or situation that no one else has thought of.

It might also be difficult to understand testing as a craft because craftsman cannot be “produced” quickly. It takes passion, commitment, knowledge and patience to become a good tester.

The definition of testing that I like the most comes from James Bach “Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.

This brings us to the question how can you spot a good tester? You will find him/her at testing conferences and local meet ups. When you have a passion you will always try to find others that feel the same way that you do. You will see them never giving up, trying to imagine the unthinkable. You will always count on their commitment. You will always see them reading books, magazines or articles about testing.

I would like to believe that once you have reached this part of the article you can see, as I and many other do, that there are differences between Quality Assurance, Quality Control, Checking and Testing.

We as testers are the last in the “production chain”. We see the final product. We see all the things that could have been better. We might receive sometimes the Quality Assurance Engineer title because once we see what went wrong we fight for it never to happen again. But this is only one part of what Quality Assurance is about.

So what is testing in the end? It’s craft. It’s a craft that takes parts of Quality Assurance, Quality Control, Checking and mixes them with “exploration, discovery, investigation, and learning” questioning, study, modeling, observation, inference.

Teamwork exercise – 8 people controlling 2 RC cars

For some time I have been thinking of fun teamwork exercise. Something that  would also involve competition and drive to succeed.

Here is what I came up with: eight people controlling 2 RC cars and racing with them, sounds like fun? Here is how I did it…..

I started with a set of RC racing cars, 8 buttons, some 15 meters of cable, a pvc pipe, duct tape, tennis racket grip tape.

raw materials

Why? so that you can re-wire the RC remote control in order to have one single button for each direction (front, back, left and right). So that it takes 4 people to drive the RC car.

change wires

After a couple of hours (here I have to thank my brother for his help) we manage to finish. Now I have 2 RC cars, that each needs to be controlled by 4 people.

final result

To me sounds like fun racing time is near.  All I need is 8 volunteers… :p and some traffic cones…




I have created my own Lego scrum scenario. How does it start?

Everything starts with a A2 paper where everything has to be build on and:

The user stories

lego user story

A box of Lego and a box with road sings and utility vehicles

lego set and box

From this point on you have to build the city during 2 releases (one of 3 sprints and one of 2 sprints).

One sprint takes 15 minutes, planning 5 minute, review 1 minute and retrospective 10 minutes. You have to choose randomly who will present in the review.

There are also pop up user stories, not just the ones above.

For most of the user stories there is box. The box will be available only when the sprint backlog contains the corresponding user story. One team member can see what it is in the box, before you add it in the sprint backlog, and then explain it to the others. This is the way I introduced SPIKES in the game.

Depending on the mood of the participants the timelines and be adjusted or other sprints added. It can take between 2.5 hours or 4 hours (the longest I have ever done).

Bug Fairy Tales

Learning from your mistakes is one thing. But learning as a group from everyone’s mistakes is powerful. There is however one catch: you don’t like you mistakes, how can you feed good by sharing them with others?

I often asked myself what does a bug want? I came up with this: it wants go into production. Some made it to the testing environment, some to the demo environment but the best ones make it to production.

So I collected some of the most important production bugs and put them in a magazine:

bug fairy tales

While working on it I followed a couple of guidelines:

  1. The title should be generic, something that resonates
    • Automating tests that were not good enough
    • Forgetting about configurations and setup
    • When something goes wrong, make sure you understand why
    • The road to bad things is paved with good intentions
    • There is never a single root cause.
  2. White it without naming anyone or do not give any proof of what happened (incident or bug IDs)
  3. Add visual highlights.
    • The road to bad things is paved with good intentions blame
    • There is never a single root cause.


4. Get the Fairy Tales from the colleges. Someone who was there when the defect was spotted. Add their photo and name as the Fairy Tale Teller. You should guide them to create the stories.

5. When possible make references to methodologies or guidelines

6. Explain to everyone so that they understand.  Draw diagrams and explanation technology details.

diagram for

7. If you consider it useful add world famous bugs.

How to turn it into a book or magazine?

Get some good printer paper. Arrange the pages so that you print 2 pages on side of a  A4 paper (you should have 4 pages printed on a single A4 sheet, front and back).  It might take some time to figure out the order of the pages. In the example bellow the Bug Fairy Tale has just 4 pages that will be printed on single A4 paper. The first page is actually the last page of the magazine.

make the magazine

Bend the paper on the middle so that you get a A5 Bug Fairy Tale magazine.

You will also need a A4 stapler in case you have more than 1 sheet. It is like a extension for a regular stapler. It will allow you to bind the A4 in the middle so that you get a A5 magazine and all sheet are together.


Build your context mnemonic

When you talk about effectiveness (doing a right thing) a mnemonic is very useful. In software testing there are a lot of mnemonics.

One of the best resources I fond is from Quality Perspectives.

There is a catch 22 with mnemonics: if you want to use them everywhere they have to be generic. If they are generic some things might not apply in your context.

I wanted to build a mnemonic for my colleagues. I used the Testing Map as a resource but also the mnemonics from Quality Perspectives.

It found  it  extremely difficult to make a easy to remember mnemonic. Sometimes finding the right combination can take too much time.

I have settled on a approach where the wording of the mnemonic has to be just a little bit memorable but in order to make sure you remember it you relay on visual memory.

Here is what I came up with within the context of my company:


How is is this more easy to remember? Here are a things I though of:

  1. CAGS – very strange word indeed. But if you look at the colors you will see that they fade because Compliance can include Accessibility,  Globalization and Standards.
  2. CRUD in Romanian mean raw, so green is a very suitable color.
  3. The icon from SSH protocol will be very easy to remember. Perhaps I will throw a broccoli in there just to make it more fun.
  4. BAD – with a picture of Michael Jackson might bring back memories.
  5. A D with a table in it it will make you remember easily decision table.
  6. CIF is a popular “chemical”. What you do with it … you DO.

The next step is to go to the team and let them tailor it.

How to learn from testing challenges

testingchallengesYou can always schedule a competition based on the existing challenges.

There is also a lot to learn by sharing, during a workshop, how to finish the challenges. Even if the number of tests might be limited how to get to those tests, how to think them is different form person to person.

Creating new scenarios is also very challenging. You need to know and understand what to test, how to test and to come up with a algorithm that validates or invalidates the inputs and actions. I found that brainstorming within a group is very useful to come up with new ideas to talk about what and how to test it.

Testing Challenges

Testing challenges is a ongoing project where I want to enable tester to test their skill.
If you have any suggestions please let me know. I am especially hunting for new testing scenarios or places where testers can test their skills.

As testers we are paid to tests and most often bound non disclosure agreements. We like to talk a lot about testing but we also need a place to practice and exercise.

Possible future scenarios:

  • Positive tests for e-mails
  • Quantity update in a shopping cart
  • Create a password