At the Ministry of Testing’s TestBash and Test.Bash() 2019 it was my pleasure to speak with people about accessibility through my quiz.
This poem was performed at Test.Bash() Manchester in their 99-second talks. This was inspired by Krissie Barrick’s talk at Accessibility Nottingham about a day in the life of a person with disabilities
A testing poem performed at TestBash Manchester 2019 and inspired by Tony Walsh’s Take This Pen
Following a post on LinkedIn about side hustles, where, in the comments, I mentioned working on an accessibility workshop. I was asked ‘What’s an accessibility advocate?’ I say it often and had an internal understanding but had never properly thought about defining it for myself or to anyone.
This was the beginning of my reply.
My definition is someone who promotes, talks about and actively brings accessibility into conversations.
That felt about right but I see lots of similarities with other disciplines and their advocates that I wanted to provide some context. A distraction from preparing to deliver my very first accessibility workshop at Sky in Leeds, but a worthwhile one I think.
So I continued;
A bit like security is seen as a pure specialism and scary by some, many see accessibility the same way. For both there is so much we can do.
• Ask if they have been considered early on in the process.
• What strategies are we using?
• Include basic tests in our exploration
Considered at the design stage instead of hard to add items they become design choices and therefore much cheaper. Considered after something is built, they are much more costly!
I was fairly comfortable with my answer now but I couldn’t quite drop the thought. I kept thinking about my immediate reply and was it really deep enough? Did I understand what I wanted to achieve by doing this as well as I assumed I did? Should I, as any good tester would, question my assumptions for a clearer understanding? I came to the conclusion, after some thought, that yes, yes I should question myself. It didn’t take long to identify that it is those things, but there’s more to it.
I make it very clear at the beginning of conversations, talks and even recently in a workshop given at Sky in Leeds and a meetup in Nottingham, that I am in no way an accessibility expert. But that doesn’t mean I can’t teach others what I know, spread the message and amplify others voices in the space. It never ceases to amaze me of the knowledge we take for granted as common place that isn’t widely known. Simple things like adding Alternate Text (Alt-text) to images accessed digitally seem obvious to allow people who need to use screen readers to navigate. But they only become obvious once you consider those users.
Over the last few years I’ve looked at different ways to spread the message.
• Invented a visual heuristic (https://www.thebigtesttheory.com/blog/2019/5/13/my-first-experiences-with-accessibility-testing).
• Shared information.
• Been a co-host of an accessibility power hour (https://club.ministryoftesting.com/t/power-hour-accessibility-testing/26064) on the Ministry of Testing Club.
• Created a quiz I believe is unique (and so far very well received) and that I’m still improving. It is deliberately very visual, as that is the target audience, but I want to make sure non-visual people can also take part by providing a fully accessible version online. Although I suspect they know most if not all of the answers already!
The next stage is developing a longer workshop to help people to conduct a basic accessibility audit of their own sites and apps. Learning this will allow those attending to have those conversations up front and (hopefully) influence the design. Essentially, I’m hoping to inspire others to become accessibility advocates themselves. I have an hour and a half workshop already and am close to submitting a half day workshop to conferences. So watch out for it coming to a conference near you soon. I’m also considering offering this to companies in house covering my time and expenses e.g. less about making a profit and more around spreading the message.
The next logical question is why? What makes someone want to be an accessibility advocate? Personal experience? Or just that it’s the right thing to do. While it is the right thing to do the best explanation I can give comes from someone else.
Four-time U.S. Paralympic medallist Tucker Dupree used to do a lot of public speaking during his competitive swimming days. His talks would often challenge the audience to think differently about people with disabilities.
I’d always open my speech with, ‘As a person with a disability, I belong to one of the largest minorities in the world, and on top of that, it’s a minority that anyone in this room can become a part of at any point in their life. You can acquire a physical disability at any point in your life, and disability comes in every culture and in every colour…’
One thing I hold to be true in everything I’ve learned is that the general perception that Accessibility = Disability is not quite correct. In the majority of cases, things that affect people with disabilities can equally affect people without. While it is important of the people with, Accessibility is more about Inclusion so becomes wider than just conformance to the guidelines.
So overall while I’m happy with my contribution although I know it is only a drop in the ocean and it needs more to convince those with the power that this is something we have to do. No easy task but I feel, a worthwhile pursuit.
Near the end of writing this blog post, WebAIM published their re-analysis of the top one million web home pages which can be found at this locations, https://webaim.org/projects/million/update
They categorise errors as;
Errors are accessibility issues that are automatically detectable via WAVE, have notable end user impact, and are likely WCAG 2 conformance failures.
The other reason everyone should be accessibility advocates is what they found. A 98% failure rate. And that’s only based on automatically detectable errors! We can and should do better, but only if we look for and call out these issues.
This post has been written as part of the Ministry of Testing Bloggers Club Sprint 13 https://club.ministryoftesting.com/t/bloggers-club-sprint-13-new-timelines/24995
The brief was; Your first experiences with accessibility testing. How you started, where your learning began and any assumptions you had to question, change or drop completely.
During my time at TestBash Brighton possibly the most common subject to pop up was related to group testing activities. Be that mobbing, bug bashes or whole team test days built into the process. Maaike Brinkhof in her talk, ‘Exploratory Testing with the Team, a Journey Worth Taking’ explained how they undertook ‘Team Test Sessions’ before they released.
This is the story of our first foray into testing as a team.
I’ve been a member of the online testing community for a long time, initially though Twitter by following some testing ‘names’ and then through the Ministry of Testing. A lot has been written about how welcoming the Ministry is and how they put inclusion and diversity at the forefront of their thinking and planning and I can only echo that. They have not only helped me improve as a tester, they have helped me improve as a person and had a massive and unexpected impact on me. They helped me be braver!
I’ve written poetry and parody lyrics to songs since I was young. These were always for me, with a couple of minor exceptions to my partner! You know, valentines and other mushy stuff… But something happened along the way and posting my first foray into testing related output (https://www.thebigtesttheory.com/blog/2017/11/26/if-ady-stokes-2017) in the club as well as my blog (which was also for me, never expected anyone else to read it) was a very big step for me. It took a long time for me to decide to share, then even longer to hit the Create Topic button. Fortunately there are some very kind people in our community who enjoyed it and encouraged me to do more.
At Brighton in my talk, Test all the Things with the Periodic Table of Testing, I had an irrational idea to create a song for people to join in and hopefully remember my table. I always get nervous before talking but it was several levels raised above normal thinking about singing (well, what I call singing) in public. Again I was encouraged by many but I need to shout out to Gem Hill and Emma Keaveny (our brilliant Essentials host) for their encouragement. To quote Emma;
Yes my friend, be brave! It’s the best place to be brave.
And it was! Despite my shaky voice, messing up the last chorus and going out of rhythm I did it! With more help from the amazing Gwen Diagram and João Proença through a Twitter request I had my support and the whole crown joined in. Warning, if you play this, you will hear my truly awful singing. No excuses, but it is a bit out of sync… honest… 😊
Lyrics below if you want to see but the story isn’t quite finished…
Two days later I was even braver in the 99 second talks at TestBash. I read out my poem, Does it Work. The very first time I have ever read out one of my poems. I was shaking badly throughout to the point I nearly dropped the mic (not in a cool at the end way like Vern Richards (love him) suggested).
Being brave can sometimes have its rewards and mine was incredible. I’ve been a long-time admirer of Angie Jones and was fortunate to sit on the same table of her for TestBash. It literally made my week when I collapsed in my chair, stunned by the reaction and that I had actually beaten my fears, and Angie high fived me and said, ‘good job’. I have had it confirmed that despite being a 52 year old the correct label for my reaction was that I had a complete ‘fanboy moment’! I make no apologies for this, Angie is a legend, period.
If you take anything away from my ramblings please let it be, be brave. I am very confident in saying that whatever it is, whatever you want to try, we got you. Me, Ministry of Testing, the whole community. No judgement, just love. Be brave.
We Will Test You (music by Queen; lyrics by Ady Stokes)
Buddy you’re a visual heuristic
Describing the breath of the testing universe
You got elements in place
Paths you can trace
It’s something you can use all over the place, Singin’
We will we will test you (come on)
We will we will test you
Buddy you have different elements
Covering how you can test all the things
You now have a place
You can embrace
You can use this table all over the place
We will we will test you (with the periodic table!)
We will we will test you
Buddy you can scope your projects
Listing all the things that you want to test some day
No mud on your face
It’s no disgrace
To help you test the things give it pride of place
We will we will test you
We will we will test you
For my talk at TestBash Brighton Essentials (see ref below for slides and more) one of the things I covered was what I believe testing is. Over the last 6-months or so I have been mentoring a new tester on their journey and to start that journey, I had to explain what testing is.
There are a many, many attempts to explain testing and I even referenced one of my favourite descriptions in my presentation.
Testing is the infinite process of comparing the invisible to the ambiguous so as to avoid the unthinkable happening to the anonymous.
Over time, the more I looked at how different people went to great lengths to explain the craft, the techniques, the mind set and from other perspectives. I though, there must be a simpler way of beginning the conversation as it is my opinion that when trying to explain what ‘testing is…’ we do ourselves a disservice and answer instead, ‘what testers do…’ or a close variation.
My belief is when we do this, we confuse ourselves and others and we would really help our craft and each other if we simply said;
Testing is part of risk mitigation for the product or system.
Now I know that might sound overly simplistic when we are sometimes put in the position of defending our profession and craft. But, we should follow up with;
And we do that by…
This way, when we talk about critical thinking, observation, problem identification and solving; explain why automation helps but isn’t a solution in of itself; offer advice about bias, empathy and inclusion; offer opinions on observability and testability and ask unexpected questions from the, ‘but that would never happen’ category, there would be a clear concept that all these things help us identify and reduce risk for the product or system we are helping develop.
I would be really happy to hear others opinions on this way of thinking. Please let me know in the comments or follow up on Twitter @CricketRulz
Talk link to be added later, may be Pro or attendees only, to be confirmed
A testing poem - Does it work?
So, does it work?
They long for the elation of validation
For their creation
It works on my machine
And the code looks oh so clean
And from the unit testing I can gleam
All is working
So it seems
So, does it work?
I’ll instigate an investigation
Think about the context situation
People of all sorts
And their variation
Accessible personas and maybe a simulation
I’ll examine the foundation
Report my observation
And if there’s any recommendation
I’ll happily give a verbalisation
Of the points in my notation
So, does it work?
I can see your animation
As I check your installation
As we run the integration
We will gain illumination
And ensure no degradation
So, does it work?
I’m not one for exaggeration
As I ponder my interpretation
Of the myriad combination
To begin theorisation
Of the behaviours sophistication
SO… DOES IT WORK?!?!
Based on my examination
And with no more procrastination
I’ll give my declaration
With a final proclamation
That it’s a veritable sensation
So to answer your communication
Asking about the classification
I have no hesitation
Declaring my appreciation
Of your wonderful formation
Based on the time allowed and data available I think we have a reasonable level of confidence to move this forward.
In November I did a presentation at Leeds Tester Gathering with the slides available here; (https://www.slideshare.net/adystokes/2017-11-leeds-tester-gathering-i-see-no-bugs) The talk was about how a test manager back in 2005 pitted testers and developers against each other but using the measure of bugs. They were good for testers and bad for developers. This lead to arguments about features, not in the requirements etc. etc. Basically a rant about attitude, human communication and language that lead to me stop finding bugs and instead just describe behaviour. For anyone who has seen my version of Rudyard Kipling’s famous poem ‘If’ written as if he were a tester you may already know I sometimes do strange things with others great work.
For the gathering I was hoping to have a version of the Fresh Prince of Bel Air theme tune to close but I simply didn’t finish it in time. So here in it’s glory is my version of the theme telling the tale of my belief that as an entity ‘bugs’ have no intrinsic value, are divisive and a hindrance to collaboration. Controversial I know but that’s what I feel. There’s a link at the end if you want to have a go singing it!
The Fresh Prince of Testing
This is the story all about how
My life got flipped turned upside down
And I’d like to take a minute just sit right there
I’ll tell you why I don’t find bugs because there’s no value there
In West Yorkshire born and raised
Finding bugs was how I spent most of my days
Chilling out, maxing, relaxing all cool
Executing scripts as a general rule
When a test manager who was up to no good
Started making trouble in my neighbourhood
I was measured on bugs and the Devs were too
I could tell pretty quickly that we were all screwed
I whistled for a dev and when they came near
I told them about bugs but it was like they didn’t hear
I wish I could say arguments were rare
But they were like mansions in the town of Bel Air
I pulled my hair out in chunks and the Devs did too
All we did was argue and the quality blew
Then I said eureka about bugs I don’t care
I’ll just describe the behaviour that I see there
So the arguments stopped and the quality grew
The manager left coz he didn’t know what to do
Thanks for taking a minute and sitting right there
Now you know why I don’t find bugs
And why my boss doesn’t care.
Instrumental if you want to sing along!
Fresh Prince of Bel Air theme song (short version from the TV opening)
‘If’ by Rudyard Kipling was written in 1895. Here’s my version, if Rudyard were a tester. Please excuse the artistic licence!
If – Ady Stokes 2017
If you can keep your head and explain the quality of the product that’s due,
When all about you are losing theirs, and blaming it on you.
If you can trust yourself when others are doubting you,
But make allowance for their doubting too.
If you are treated worse than you really should,
Or there’s misunderstanding of what you said,
And that the future doesn’t look too good,
Or being told that your art is dead.
If you can advocate your worth and believe in testing’s value,
And think of every possible practical scenario that could transpire.
If you can dream of all the things the customer might do,
Then analyse the results to take your testing higher.
If you can meet with Triumph and Disaster,
And treat those two impostors just the same.
If you can script, but not make scripts your master,
And make discovery of truth your aim.
If you can force your heart and nerve and sinew,
To serve your turn long after they are gone.
And so hold on when there is nothing in you,
Except the Will which says to the team: 'Hold on!'
And explain simply your deductive and inductive reasoning,
That was led by curiosity and guided by your intuition.
If you can hear bias arguments but smile and keep on listening,
Then use soft skills and diplomacy to keep everyone on mission.
If you can talk with crowds and keep your virtue,
Or walk with Management - not losing the common touch.
If neither foes nor loving friends can hurt you,
If all stakeholders count with you, but none too much.
If you can fill the unforgiving minute, (or 99 seconds)
With sixty seconds' worth of testing value.
And find a great community who’s in it,
To share their knowledge and is truly diverse too.
If you can treat everyone equally,
No matter their difference to you.
If you can treat their gender, origins and history,
As valuable to offer a different view.
If you can be an ally to all those who need,
And defend their right to be just who they are.
Support everybody’s rights to succeed,
And mentor their journey so they may go far.
If you can see scenarios that will make the testing great,
As well as the risks that others do not spot.
If you can see the value in which tests to automate,
And more, the value in which to not.
If you can use tools to add value to your quest,
But understand their value to aid and not replace.
If you’re not afraid of change and always do your best,
And can interrogate a database.
If you can be the personas and advocate for all,
And the voice of the customer in use and value too.
And advocate and test for your product to be accessible to all.
Explore to make discoveries because that’s what you do.
If you accept that those discoveries,
Will change what you thought you knew.
Look for threats and system recoveries,
And help defend against those too.
And then know all these things here,
Are just a part of what you do.
And be brave and show no fear,
Accepting our learning will never be through.
If your passion for testing means that you will never quit,
You will be here to the end.
Yours is the Earth and everything that's in it,
And - which is more - you'll be a tester, my friend!
Categorising the Periodic Table of Testing to help scope projects
There are several reasons, some more personal than others. Entering my 52nd year of visiting Planet Earth my mind needs as much help as it can get. I’m not saying I’m over the hill, but hills can be steep! I have noticed the odd occasion where words escape me so best not to take chances on forgetting vital information.
The Periodic Table of Testing has elements. But not all elements are created equal! Elements can be useful for specific projects but not appropriate in some cases for lots of different reasons. Therefore it’s not going to be something that comes into consideration when thinking about project scope or test strategies too often but could be considered depending on the project and context. Things like accessibility, test data and access however are areas that must be primary considerations whenever you start something new. There are others that should be considered and that’s why I wanted to categorise the table both to help start those conversations and act as a bit of a guide without dictating.
I’ve always believed that mnemonics and heuristics are excellent helpers for tasks in general and more so for repetitive ones. Things we do over and over again have an annoying tendency to become brittle. We forget one little step or thing and then it quickly becomes ingrained as if we always did it that way. Having useful reminders around is helpful and especially ones that look cool when put up on a wall.
What’s in each category?
So below is the first draft of Scoping Categories which I have classed as considerations. It’s important to say that everything done on both the table and these categories are my opinion from my context. I don’t expect everyone or indeed anyone to share my opinions so, as with the table itself, you may look at this and think, ‘that’s so wrong’ and you would be right! – In your context. While I feel this can be a useful reminder and tool, I believe the power is in its ability as a useful guideline and being that reminder to help start conversations by prompting questions such as:
· What does Security mean to us in this situation?
· How can we achieve living documentation for our project?
· Are our risks defined enough to prioritise testing?
· How can we use automation for testing?
· Do Digital technologies apply?
· How do I start testing for accessibility?
I’ve used a Must, Should, Could category definition in an attempt to reinforce the thought that these are considerations, not instructions. While I have considered each item carefully and where I think they fit, I don’t think it would be useful to go through them all and explain why they are where they are. For one you would be bored to death and this blog piece would be far too long! But I do think I should try to explain a little why I believe you must consider those under that category. I’ll (try to) make it brief, I promise.
Accessibility – Number one consideration from design principles to user base and the most overlooked. Of course it’s there!
End to end – Focused testing is vital but if you don’t test end to end (whatever that means to your project) you can’t be confident everything ties together.
Discovery – It’s important in any plan or strategy to remember you will discover things that change your plans (put in Oscar Wilde quote, life is what happens when you’re busy making plans, for talk)
Exploring – I extolled the virtues of exploratory testing in this post so won’t go on again but this also covers general exploring of the system you will be testing. Things that were not apparent from your up front information will be discovered.
User Access / Permissions / Roles – If you think about how users will be managed you can’t help but think of the different states for your testing. This will also give you an early indication of whether Personas will be useful.
Security – Linked to the above, even purely internal applications have potential security issues that need considering.
Analytics – From usage statistics, usability analysis and many others understanding how your system is being used and working will help direct your testing focus and efforts.
Living Documentation – Lets face it, on one hand you have to document, on the other automated tests can be really valuable. Living documentation gives you the best of both done well. Lightweight but always up to date documentation. Focused automated tests on the most valuable functions your system performs. Win-Win!
Unit Tests – Almost goes without saying.
Testability – Of course! Even in the most technical builds we must ask, how will we test this? Harnesses, scripts to reset data, fake services, it doesn’t matter what you use or how you apply it, it has to be testable or you have no idea what your releasing. (in the talk point to Ash’s blog and other good sources)
Test Data – In context (yes I said it again) possible one of the most vital ingredients for testing. Whether it’s unusual, short or long names (link to a cool article), different statuses or variants in information test data can be your best friend. Just remember to scramble it if it’s a copy of live!
Monitors / Logs / Audit trails – We all love a good audit… don’t we? Having logs and audit trails makes audit easy but having access to what your system is up to can help identify the strangest of behaviours. Logs have so much value, find them, understand them and make sure your team gives them the priority (in context!) they need.
Risk Based Testing – aligning test plans to risk is a way of ensuring your testing is adding the most value to the project at any given time.
I excluded personal elements and methodologies on the basis that those are related to ways of working rather than tools to use in plans or strategies. I’m happy to hear a different opinion though. For those who are eagle eyed yes the ‘first’ is version 1.1 as I made a number of formatting changes and tweaks while writing up and it’s for my own tracking.
Note: The categorisation above is based on the latest version of the table, 1.7 which is below and on the Current Periodic Table of Testing and Archive
While I make absolutely no claims to be anywhere near the level of a genius like Edward De Bono I've found adding my own 'hat' to the Six Thinking Hats
(http://www.debonogroup.com/six_thinking_hats.php) to be useful in making the technique more relatable to modern applications.
Introducing, the 'Hard Hat'
De Bono’s 6 hats is something I return to regularly. Some time ago I added a 7th ‘hat’ that I feel brings an element of the modern digital/mobile world to it.
A purple, hard hat to represent where the work/workloads are. For example: How is memory and CPU usage affected? What puts the most work on the system and needs to be monitored? I believe that this assists me to think about things in a more digital/SCAMI technologies way when using this technique.
I did try researching if anyone else had added their own hats and there are some variations with a gold hat for customers and a grey hat for consequences/cycles but nothing I've found like my purple one that pulls the digital/mobile world into the technique.
I've always found the mind map below useful when applying six hats and have added the purple hat. I hope Paul doesn't mind! I also hope someone else might find this useful. If you do please let me know. Thanks.
The history of the Periodic Table of Testing with release notes.
SB – Session Based Testing -
Sub section – Approaches
Since Jonathan and James Back (satisfice.com/sbtm) documented their Session Based Test Management approach combining exploratory testing, risk management and ‘management’ oversight there has been a lot written. Hopefully by now most people know the benefits of exploratory testing and some of the various methods of recording that activity. In this article I hope to share a brief overview so we are on the same page. A list of the main benefits and some minor drawbacks. And finally the question technique I apply when using this in my day to day work.
The Session Based Testing approach was developed for a project to allow their test team to ‘organise their work without obstructing the flexibility and serendipity that makes exploratory testing useful’. They needed a way to keep track of what was happening, so they could in turn report back to a ‘demanding client’, while ensuring the time spent created the biggest return on investment.
Essentially this is structured
to help organise thoughts, capture questions and insights and allow rapid feedback. Key elements to this approach include;
- Each session is chartered (associated with a specific mission)
- Uninterrupted (as much as is possible)
- A template is used to record the details of the mission and findings
- Reviewable ( a ‘report’ is produced to document findings and questions and the tester is ‘debriefed’)
- Time-boxed (with flexibility but generally no more than 2 hours)
In my opinion, there are a number of flexible points in the approach and tips that are worth being aware of, especially if you’re doing this for the first time;
- I don’t think it matters if you call it a charter, mission or focus. As long as you generally stick to your subject, although picking one might help when sharing for consistency.
- Interruptions should be avoided if possible. On occasion I’ve shut down Outlook and put my headphones on for these types of sessions. At one time I even had a red flag on my desk which indicated do not disturb unless it was urgent.
- There are templates available or you can create your own like I did. Again it’s useful for consistency to stick with one you’re happy with.
- Reviewable. A lot of focus is on ‘management’ reviews but team, peer or even self is fine, as long as what you find generates actionable insights rather than getting filed away never to add value.
- Time-boxed. If you start small with something very specific that’s a good way to get a feel for this technique and learn to focus. I can sometimes be like the dogs in ‘Up’ and be distracted by squirrels! Learn to note where the squirrels are and why you need to look at them later.
Question technique and template:
I admit that I often use this as a mental reminder, rather than something to populate, as my preference is to speak to a developer on my team immediately after a session to investigate or question. (I don’t raise bugs I describe behaviour and in writing this, that’s probably what my next post will be on. I’ll add a link to it on here when done.) Only if this isn’t possible due to availability will I actually fill things in from the notes I have taken during sessions. For me, this is a disposable document with a short shelf life used to capture, discuss, resolve (or not), and most importantly discard.
I’ve reproduced the template in bullet form rather than embed a PDF or word document, that way I hope it will be easier for you guys to take away and make your own. When you get to the questions you might find, as I do quite often, you will remove some before you start as not applicable, or you won’t have filled some in when you’ve finished. It’s supposed to be flexible like that but you should take a moment to understand why they are not populated or applicable to the session as that may prompt some other thoughts.
- The Basics: Date; App/function under test (brief description); any other useful information depending on your context
- Any dependencies vital to the testing (connections, files, data, hardware etc. this helps make sure you have them before you start)
- Any information that is useful such as material/learning’s from previous sessions, personas to use, environments, tools etc.
- Test strategy (a consideration of techniques you might use as a flexible plan is often more useful than no plan, but don’t be afraid to improvise as that’s half the fun and discovery may make your plan obsolete quite quickly)
- *Metrics (see rant at the end of this post)
- The questions: (with a brief reasoning for them)
What do I expect? (even if it is something brand new I always have some expectations)
What do I assume? (sets a context that I can query as I go)
Are there any risks I should be aware of? (to execution, the system, helps anyone else reading have context)
What do I notice? (behaviour; usability)
What do I suspect? (things that I feel, not always based on facts but that I don’t want to lose)
What am I puzzled by? (behaviour that doesn’t feel right)
What am I afraid of? (high priority concerns about the item under test)
What do I appreciate/like? (always good to have some positive feedback)
Debrief (originally between the tester and a manager there’s a checklist of questions on
. My version is more often a conversation with the developer with questions or queries, but can also be with the product owner or stakeholder depending on what I find/context. I’m not saying don’t do this, rather do it only where it’s going to add value.
This post is getting a bit longer than I’d hoped but I feel it’s important to summarise the benefits and possible drawbacks of using this method so there’s a balanced view.
Allows control to be added to an ‘uncontrolled process’
Can be harder to replicate findings as full details are not captured
Makes exploratory testing auditable
As a ‘new’ technique it has to be learnt
Testing can start almost immediately
Recording exploratory testing (rather than brief notes) can break focus / concentration if you're more worried about doing it
Makes exploratory testing measurable through metrics gathered
Time is required to analyse and report on metrics
Flexible process that can be tailored to multiple situations
Time is required to discuss/give feedback to potentially the ‘wrong’ person
Biggest and most important issues often found first
Can help explain what testers do to clients, stakeholders and the uninformed
Given all the above, if you have to justify exploratory testing, (notwithstanding you should be looking for a new job!) then using session based test management either in its original form or some hybrid version could be the convincer you’re looking for. In the table above, ‘management’ will generally only see the Pro’s column which covers a lot of the things ‘they’ will worry about. But seriously, look for a new job!
*Metrics: I personally don’t think these are useful for virtually anything (oh, more controversy!), but if you absolutely have to report back to someone, a manager who knows little or nothing about what testing really is for example, here are some metric examples. How much time you’ve spent such as start/end times on actually executing testing; blocked; recording findings; actionable insights; questions/queries; potential problem; bugs; obstacles; screen shots or some other method of recording or documents to show any issues to help replicate them.
- As a Tester, how do you grow to keep up with the current trends in testing and development?
- As someone responsible for leading Testers, how do you help Testers grow?
- Generalising skills can make you more valuable to a company
- Generalisation shouldn’t be at the expense of your ‘deep’ skill
- The role of ‘pure’ specialist is still valuable (E.g. Penetration; Performance; UX/UI; Accessibility)
- ‘Pure’ specialist as a consultant or service was expressed as the most powerful use