Featured

Postman Flows: A Block Reference Guide

In my earlier blog post, I talked about Postman’s latest beta feature, Flows. This feature allows you to chain requests in a low/no code format. To build your own Flow, instead of using complex JavaScript in pre-request/test tabs, you move widgets called “Blocks” around on a canvas to allow you to easily visualise what is happening to the information as it Flows. For example, if I create a user, I can take the ID from the response body of my API call and pass it into my next request, loop or variable. Think of it a like using lego or duplo blocks to build a bridge from one place to another – your bridge is a lot more stable if you link the blocks together than if you simply place them side by side.

TLDR: You can find a great living GitHub page from the Postman team that describes a lot of what I am saying below here.

How can I access the blocks I need to create my Flow?

There are currently a few ways to view and add blocks to your Flow.

  • After creating a new flow, hit the “Add Block” plus sign next to the orange Start Button at the bottom of the canvas. This allows you to scroll through the blocks. When you select a block, it will create a standalone widget, that you can move to wherever you like in the Flow. Useful for when playing around or laying out larger flows.
Add block icon used to generate a Block list
  • After creating a new flow, a block list containing all the blocks you can select is auto-generated and connected via a “noodle” link to the start button. This allows you to get started on your flow straight away.
Example of clicking Event to the right of start button to generate Block list
  • Pressing the blue plus (“+”) icon to the right of an existing block will bring up the block list and allow you to continue your flow based on the outcome of the previous block.

Here is a quick vid to demonstrate all of these methods. Super quick and easy right?

But what do each of the blocks actually do?

Here is my brief reference guide. I’ll try to update this as and when Postman releases updates – which they do regularly. In fact, their latest block, named Check, was only released just this week. Exciting times!

Once you’ve found the block you need, either scroll through the blocklist or click into the Search field and begin to type the block name. Clicking on the name will create that block, that you can customise with input and/or output data to build your flow.

Block NameDescriptionExampleInputsOutputs
Send RequestPerforms a GET/POST/DELETE etc. Send a GET Booking request from API collection Restful Booker using Environment “Production”VariablesResponse
Test
ConditionMake a condition check and fork the data to either accept or reject.If the status code is 200 AcceptDataAccept Reject
CheckPass the data through if the check passes. Can take inputs from 2 sources.If the BookingID does not exist AND the ping generates 200 then pass the data to the next block. Primary SecondaryPrimary
TerminalShows output logs. Note actual log data generated shown in terminal block top right hand corner of canvas.End of the line. Show all logs of all request that have made it through the flow.Data
Assign VariablesAssign data to variables and pass it to a requestTake the “token” value from the response body and assign this to an existing variable named “token”. DataVariables
Create VariablesGenerate a new variable(s)Take the userId from response of POST user endpoint and create a new variable called “newUserId”TriggerVariables
Parse JSONConvert text input data to JSON treeThe equivalent of var jsonData = pm.response.json();
Allowing interrogation of response body data.
DataJSON
DelayWait for specified duration before passing data throughWait for 300[milliseconds] before checking status code in order to allow for integration points e.g. database query, third party API.DataData
Test SummaryAggregates all assertions to generate a report. Note actual test summary data generated shown in test summary block top right hand corner of canvas.Tests Passed: 20Test
For EachPerforms a for each loop to iterate through each item in a listFor each ID in the response body, do X.DataItem
ConcatenateConcatenate two data packets together into a listTake all the users from endpoint A and add them to a list of all the rooms from endpoint B to make List C.List (data from first source)
Data (data from second source)
List
MergeMerge the source Object to the target ObjectTake all the users from endpoint A and merge them with all the users from endpoint B and output all the unique users into list C.Target
Source
Data
List – PopExtract the first element from the listGet the users from list C and “pop” the first user. Do this with the first user and do that with the rest.DataFirst
Rest
Group ByGrouping elements in a listTake list C and output all the users who have an ID from 1-100 into a group called “first_100_users”DataGroup
Table showing each Postman Flow Block, alongside a description, example and input and outputs.

I hope this has helped to get those juices whirring and thinking up all the ways you can use the list items to create some simple, or more advanced Flows. I am far from a Postman expert, so am always happy to hear feedback on changes to the content – if it doesn’t make sense then I want to hear about it!

See you soon, thanks for reading.

Beth

Featured

Postman Flows: A Guide

If you prefer to listen to an audio version of this blog, check out this racket instead:-

https://racket.com/beth_askher/r6E8Y

In the past few weeks, Postman have released a Beta version of a long awaited new feature, which they are calling Flows.

TLDR: look at the YouTube vids at the bottom of this blog post, they’ll tell you all you need to know. This includes a brand new video showing how to “chain” request data together to pass variables between requests. Enjoy!

What is it?

Here are the words of Postmanaught Neil Studd, on the fab new Tech Team Weekly podcast:-

“We’ve done a soft launch for a new feature called Flows, which we’ve been working on for ages. Basically it is a widget based canvas that you can drag and drop components and link them together.

Previously if you wanted to start chaining requests at Postman, or start carrying values between requests you had to know a fair amount of how to structure JavaScript and work with code.

Flows is basically our codeless approach to chaining processes together. It really opens up Postman to a whole new audience.”

Now I have been waiting for this to drop for awhile, partly out of sheer consumer curiosity as someone who is still reasonably inexpert when it comes to code, and partly because I had originally wanted to include a chapter on this feature in my Test Automation University Postman course. I think any technology that wants to make it easier for non-tech people to get into and not harder gets my vote, so this is an exciting area for Postman to move into.

I’ve had an hour or so to take a look at the beta, and here are my thoughts, as well as tips on how to use it.

How do I find Flows?

Firstly, Flows can only currently be found in the Web version of Postman. If you are using the desktop app, you can easily login to the web version should you wish to check this out.

UPDATE: Flows can be found in both the Web version and the desktop Postman app.

To access Flows:-

  • Select a workspace
  • Select Flows from the left hand horizontal menu (underneath History)
  • Click on “new flow” to view the canvas

Click on the “flow” menu on the bottom of the left hand side of the screen.

YouTube video showing where to find Postman Flows

How Do I Get Started Using Flows?

It is very important to say that this is only a BETA version, and inevitably lots of changes will happen between now and the final formal release. I have to be completely honest here and admit that it wasn’t the most intuitive feature for me to get to grips with, but nor was it so painful I had to give up on it completely. I’m sure by the time you come to read this Postman will be several iterations in and it will look and feel much better.

If you wish to explore for yourself, the Flows work by adding blocks in a chain. Think like a daisy chain necklace. Currently these blocks can take several forms in Postman:-

list of block types in Postman FlowRunner Beta

If you want to learn more about what each of the blocks do, read my other post here.

There are tonnes of use cases for this that I’m sure your brain is whirring with already, but I’m going to start with a basic example.

Say you are using an endpoint that returns a list of bookings for your hotel. You want to see if there have been lots of bookings, or its been a slow day and you haven’t had much success.

With PostmanFlows you can use blocks like “send request”, and then add a chain to a “for each” condition – this allows you to iterate through the response and run a query against it using a third block called “validate”. Finally, as the validate outputs a true or false data, each of those records can be sent to a different “terminal” log group, allowing you to view what data has been applied to what “pot”.

Picture showing an example Postman Flow

You can see this for yourself in much more detail in this YouTube vid:-

YouTube video entitled “How to use Postman Flows”

And here is another vid on how to chain requests together to “flow” data from one request to the next.

YouTube vid titled “how to chain requests using Postman Flows”

I hope this very basic guide to this new feature is useful to you. I’m looking forward to what future iterations bring, and will aim to update this blog post with future updates.

Enjoy!

Featured

Creating A Test Automation Portfolio Episode 6: C# Solution For Ui Automation Using Selenium 4.0 (aka the missing episode)

Intro

This is the missing episode. The one that I wrote and somehow inexplicably deleted (even from the trash). Rewriting as faithfully as I can the final project in my test automation portfolio series.

This time, its the one I put off until I was too far gone into the project to turn back. The one I was most nervous of attempting. Selenium 4.0 with C# to automate at the Ui level.

The Project

I used the amazing course by Carlos Kidman on Test Automation University as my starting point, and added a few additional bits and bobs along the way. I’ve since re-used this framework for another paid project, so proof that putting the effort into an automation portfolio pays off on many levels, thanks past Beth!

So, what does it do?

The framework uses Page Factory Model (POM derivative) to organise the code. It is written in C#, and a base class to store the startup/teardown info. The tests check basic functionality on the Opencart website. But I converted the version of Selenium to be the latest big upgrade, version 4.0. This meant I was itching to trying at least one of its new features. I went for relative locators:-

This allowed me to click a button by its relative location to another button. A little flaky in practice, but handy to know for future.

Here is a video I made that goes into more detail:-

Video of how to use relative locators in C#

Rex Jones II has a brilliant set of video’s which go into way more detail on all the other cool features too.

Key Features

My strategy was basic, namely:-

  • Get the original framework running.
  • Upgrade Selenium
  • Add in a drivers section so that the test can run against multiple drivers
  • Convert the tests so they run against the Opencart demo website

Here is a video of how I overcame the difficulties of installing the Edge Chromium browser – not as easy as you’d think, and judging by the fact its my most popular video I reckon a fair few people agree with me!

YouTube video of how to run tests using Microsoft Edge Chromium Browser and Selenium 4

The Code

I’ve published my repo as a template, which means you can use it as a basis to begin your automation framework. Check it out here if using the drivers is your thing:-

https://github.com/askherconsulting/CSharp_.Net_Selenium_Automation_Portfolio

However, I’ve since revisited Carlos’s course, and added a lot more of the bits I missed out from the first framework in this new upgraded version here (so I’d suggest you use this one, unless you want the drivers in which case look at the other one, I’ll update this when I have time):-

https://github.com/askherconsulting/Mailinator_Selenium_Automation

Lessons Learned

A lot of people still default to using Selenium and Ui automation tools like it. We know that isn’t always the most robust way to automate, but once you have the framework up and running and just want a quick and dirty test, sometimes it is your best option.

When I finished this particular framework, it meant that my test automation portfolio that I’d set out to do 4 months before (target being 1 year) had been completed. I was ecstatic, and continue to be proud of this. Others have read these blog posts and been inspired to do something similar – I hope you may be one of them.

T’ra for now!

Featured

Testing Pearls Of Wisdom

As you may or may not be aware, I’m currently half way through helping to deliver a software testing bootcamp. This involves me working as part of a team of instructors (who are all working testers) to give sessions on various topics to those wishing to pursue a career in software testing.

I’m enjoying the course a lot, as aside from learning loads it has made me reassess some of my core values, and try to put them in a pithy, easy to read way that can translate to different audiences.

I decided to create Twitter/LinkedIn posts using tools like Canva and Piktochart to create the images below, and I’ve been pretty amazed at the positive response.

Personal Brand Vs. Personal Reputation

My philosophy on growth

"If you learn the why, you can always Google the how"
“If you learn the why, you can always Google the how” quote

I’ve learnt this from years of watching developers and other testers, whilst also letting my Imposter Syndrome talk me into thinking that doing exactly as they were doing somehow translated into me being rubbish at my job. I attended a mock interview recently, and one of the responses was “we don’t expect you to have the answer to your fingertips every single time, just tell us your thought processes to how you know where to find the answer”. I Google stuff every. single. day. I couldn’t debug an automated test without using Stack Overflow, and I lean on others for guidance and support both in work and outside it. And this actually makes me better at my job – because often my instinct is proved wrong, and I can refine what I’ve done long before I’ve taken myself down a rabbit warren of mistakes. And judging by the 200+ comments on this post, a helluva lot of people feel the same.

Automated Car Analogy

Quote "Automated software testing is like driving an automatic car.  The car takes care of the checks, like wipes, lights and gears. This frees up the driver to focus more fully on the unexpected and more complex twists, turns and bumps in the road."
Test Automation Analogy: Automated Car

I came up with this one when driving my car (my first ever automatic, nothing fancy mind) back from the supermarket. I noticed a car in the opposite lane with its hazards on, and it took me fully 5 seconds to realise it had stopped on the road so that the driver could visit the Tesco over the fence. Not a great bit of driving, and the cars behind were similarly confused. I thought to myself, if I’d have been thinking about that for 5 seconds and had to concentrate on the road as well as gear changes and wipers (it was raining) I’d have probably stalled. But my automatic car took care of those checks for me, giving me that head space I needed to assess the dangers, or lack thereof, in front of me.

Automated testing is just the same – get those repetitive regression checks taken care of by the machine, and allow the driver to explore the application at leisure.

Bug Reporting = Archaeology

Quote "Reporting software bugs is like digging for dinosaur bones.  The best thing you can do when you find something is to uncover as much as you can before you confidently shout "It's a T-Rex!""
Bug Reporting as Archaeology Analogy

I’ve got to thank @TheTestingPeers for this one, as they refined my original thoughts. This one works on a few levels, as identified by David Maynard and Russell Craxford. Firstly, in archeology people dig in teams of different experts – so you can call on the right person to help you figure out what you are seeing as part of your analysis.

Secondly, archaeologists have to document the position of what they have found, in much the same way testers do (e.g. this is on chrome version 88.39393) – adding this detail instead of just shouting “I’ve found something”, when something may turn out to be a chicken bone will save the team time and earn you a reputation as someone who’s opinion can be relied upon.

Thirdly, archaeology, just as testing, is mostly about what you don’t find. You can asses the quality of a piece of software by its absence of bugs (as long as your testing the right things) as much as by the existence of them. Knowing something isn’t a T-Rex, and being confident of that assertion, is just as mighty.

Summary

I hope you’ve enjoyed these pithy analogies. Feel free to share with anyone who would benefit from seeing them.

Featured

Creating A Test Automation Portfolio Episode 4: API testing for Restful Booker using Postman

Intro

This blog continues my journey of creating a test automation portfolio, which has been ongoing for a few months now. Although I’ve covered API testing in an earlier post, I wanted to include Postman as the tool of choice, partly because its something we use at work, partly because I want to get better at it and partly because its such an industry leading tool.

I’ve already had a go at using a Newman call to run a Postman collection through the Microsoft DevOps Azure CI Pipeline – you can check out the code for that here as I won’t be referring to it again in this post. I find terms like DevOps and CI a bit scary when I read them in blogs, and I don’t want to put you off reading on, Postman is a super simple one to start with I promise!

Postman Image

The Project

I continued my magpie tendency and avoided reinventing the wheel by Googling “Postman tests for Restful Booker API” and hit gold with an amazing GitHub Repo from Joanna Denni. She had created a really great and easy to read repo with some excellent README instructions.

The great thing about the pretend hotel booking website Restful Booker is there is a limited number of really clearly documented API endpoints, so its easy to get a good amount of test coverage, and practice testing GET, POST, PATCH, PUT as well as DEL calls. Mark Winteringham (its co-creator) recently tweeted about the sheer number of requests which are sent to the public server, so there must be plenty of other people learning in this particular sandpit.

Tweet from Mark Winteringham About the Popularity of Restful Booker

Key Features

I wanted to develop what was already in the repo to support my learning by:-

  • Install the original repo, and get it up and running.
  • Get the existing tests to run (very straightforward, also includes pre-request script examples which is v. cool)
  • Add in a few new tests from scratch including MOCKs, randomised response variables and params
Example set of tests for a GET endpoint and below actual code snippets (for improved accessibility)
pm.test("Status code should be 200 OK", function () {
    pm.response.to.have.status(200);
});
pm.test("Response Content-Type should be json", function () {
    pm.response.to.be.json;
    pm.response.to.have.header("Content-Type", "application/json; charset=utf-8");
});
pm.test("Response should contain at least one bookingid", function () {
    var jsonData = pm.response.json();
    pm.expect(jsonData[0].bookingid).to.not.be.null;
    pm.expect(jsonData[0].bookingid).to.be.a('number');
});

Here is a video of the tests successfully passing against the public Restful Booker site when executed via a Postman Collection Runner. I did 2 iterations which gave me 130 test passes in just a few seconds – not bad eh!

Video of test execution using Postman’s Collection Runner

The Code

I’m still learning the inner circle workings of GitHub, so I have a lingering doubt that I’ve gone about this in the wrong way, but this time instead of forking my own branch I took the original code and generated pull requests for my additional work. The code can be found here:-

https://github.com/joannalaine/postman-restful-booker/pulls

Pull Requests demonstrating the additional work I have added to the original brilliant repo

Lessons Learned

I think the idea of “API” testing is massively offputting to a lot of people, because it feels super techy and “backend”. In reality, particularly when using Postman, I’ve found it way simpler than a lot of UI type automation. It is also very maintainable once you know what you’re doing with variables etc. Feel free to use this code as an example to get you started!

The other thing I’ve found about Postman that I love is the amazing amount of online help available. The Postman learning center is great, and a big shout out to the incredible Danny Dainton, who has helped me out a tonne too.

I’m now over half way through my Test Automation Portfolio, which is a pace I’m really happy with. Yaaaaaaaaay.

Onwards!

Featured

Creating a Test Automation Portfolio Episode 1: Setting a Goal

If you are interested in hearing an audio version of this article, please check out this Racket.

Not for the first time, I’d like to give my thanks to Angie Jones for her post “10 portfolio projects for aspiring test automation engineers“. Reading it made perfect sense to me, so much so that I wondered why I hadn’t thought about creating a portfolio before.

If someone were to come to an interview for a testing related position armed with their own GitHub repository containing samples of their work I can only think this would be a positive thing, for the following reasons:-

  • Even if it isn’t perfect, it gives conversations about automation a jumping off point – “can you tell me why you did X” and “what would you like to do to improve Y“, “when wouldn’t you use automation for Z” etc.
  • It demonstrates an enthusiasm and passion for testing
  • It allows a nervous interviewee an opportunity to show their skills without just speaking about them
  • If they don’t yet have commercial experience in that area, it is a great way to demonstrate what they could do if given the right opportunity
  • There may be some useful code which (assuming it is the property of the person who wrote it and not copied and pasted from someone else’s repo) may be applicable to the work-related project and the ideas can be quickly lifted and shifted to add value.

Now, lets be clear, I have only been working in automation for a few years, so I am still learning and would still class myself as a beginner/intermediate level in lots of areas. But I’m also keen to take advantage of the free online courses and resources aimed at people just like me, so I’m setting myself a long term goal with this activity.

Goal – in 1 years time I will attempt to have the following on GitHub:-

  1. C#/Java solution for web browser automation for Opencart using Selenium WebDriver
    Login
    buy an item (interact with multiple pages) – ideally atomically by API calls
    Use POM and clean code
  2. JavaScript solution for web browser automation for Opencart using Cypress
  3. API testing for Restful Booker using Postman
    Newman calls
    use GET, POST, DELETE methods
    Use parameters, environment configs, request bodies and tests – learn about mocking and try using that
  4. CI integration testing using TAIKO to read programatically from an external data source
  5. UIpath to do RPA testing preferably reading from a sql server express database to populate values to somewhere in the test
  6. BONUS: NUnit Testing of Restful Booker
  7. BONUS #2: Codeless UI Automation with TestProject

Wish me luck!

Photo by Markus Winkler on Unsplash

A Tester’s Takeaways: Postman State of API Report 2021

Postman have recently released their huge API report. It gives a long term view on the world of API’s, and includes insights from all Postman users – incredibly over 26,000 of them who responded.

I have previously blogged about a few similar reports, and what they tell me in my role as a software tester:-

What did I learn from the Postman Report?

The Postman report was interesting, because it was aimed at all users of API’s – the majority of whom are developers. You can read the report in full here: https://www.postman.com/state-of-api/#key-findings

Here are my three takeaways:-

Takeaway #1: Postman isn’t just for ad hoc manual testing anymore

Extract from Postman’s State of API Report 2021

Whilst a lot of people do still use Postman for ad hoc/exploratory testing – and as Amber Race’s seminal Test Automation University course shows, it is brilliant for that – its now clear that automating API’s isn’t seen as an edge case or extreme thing anymore. Among all respondents, 11% of time was spent on automation, but amongst Quality Engineers it was a whopping 31%. This also adds weight to the assumption that testers are responsible for automated testing in the majority of organisations, something which the other reports (mentioned above) seem to back up.

Takeaway #2: Its a man’s world

I’ll just leave this here….

Extract from Postman’s State of API Report 2021 showing 86% man, 10% woman, 1% non binary, 3% prefer not to answer

Now, this might seem high to you, but its actually double the number of women who responded to the Stackoverflow 2021 Deveoper Survey.

Extract from Stack Overflow Developer Survey, showing 91.67% man, 5.31% woman, 1.75 prefer not to say, 1.42% non binary and 0.92 in your own words

There is clearly an issue here which is tech-wide, and bigger than one company to fix. But in fairness, I think Postman do a great job at encouraging female voices to be heard, and I look forward to seeing how they seek to chip away at this imbalance in future. (*whispers* It’s actually my next career ambition to present f2f at a non-UK conference, maybe Postman Galaxy would be a good way to pop that particular cherry? 🧐#justPuttinThatOutToTheUniverse)

Takeaway #3: Remote Working Isn’t Going Anywhere

Extract from Postman State of API Report

This is one of those responses that asks more questions than it answers – I’m intrigued! It’s really interesting to me, because with 26000 replies its probably one of the most accurate bellwethers we have for tech as a whole in relation to this issue – most companies have API’s now, so its safe to assume the companies we work for fall into these categories.

I guess I would have liked to reverse the question to find out if the respondents expected to go into the office every day or at least one or more days per week (as opposed to working from home during that time). There is a big difference between being fully remote and going into the office 4 out of 5 days. It wasn’t the focus of the survey, but I’d also love to know how keen the respondents were to go into the office/work from home – my personal experience is that the younger (and often more inexperienced) folks prefer the occasional office day as its easier for them to pick up things, but its a jump to assume that’s what these Gen Z-ers thought.

In Summary

I’d recommend taking a squiz through the survey yourself – it is super easy to absorb (props to whoever put those graphics together). Its pretty clear that API’s aren’t going anywhere, are truly a global thing, and that Postman is by far the most popular tool in the market.

If you want to learn more about the features of the Postman platform, take a look at my free Test Automation University course.

T’ra for now!

A simple guide to upgrading to Selenium 4.0

What’s been going on?

Yesterday (13th October 2021) was a big day in the Selenium world. 4.0 has finally made it as a stable release.

tweet from Jim Evans announcing Selenium 4.0 Release

What does this mean for me?

According to the recent Mabl state of devOps report, Selenium is the tool of choice for the overwhelming majority of survey respondents using a functional testing tool:-

Extract from Mabl’s 2021 State of Testing In DevOps survey showing 41% of respondents used Selenium

If you already have Selenium and are wondering how easy it is to upgrade – the answer is, pretty easy really. Don’t be scared of it. It makes sure you’re using the latest code, plus there are a bunch of cool new features in this version for you to try, so upgrading makes sense.

That being said, I wanted to point out a few things that caught me out while attempting to upgrade from Java Selenium 3.1.14 to 4.0 today.

Where are the best instructions on how to upgrade?

I found the instructions on the Selenium site itself very helpful:-

I also used the Applitools blog post, how to upgrade to selenium 4.

One thing that perhaps is obvious to people who play with this stuff all day but wasn’t to me, was the missing reference to pom.xml. Perhaps because other setups use a different filename to hold their Selenium dependencies the articles weren’t explicit, but as a Maven user, in order to upgrade to Selenium 4.0 you need to go to the pom.xml and find the Selenium Java dependency and update the version like this:-

step 1: open the pom.xml (maven users only)
Step 2. change version

After you save this file, you may see some errors. These are visible in Eclipse as problems, or will appear if you run “mvn clean compile” from cmd line from the same folder as your pom.xml.

The selenium instructions include some more detail on this if you get stuck, but my errors were all to do with Desired Capabilities. From what I understand, this is essentially deprecated in Selenium 4, because it isn’t W3C compliant.

To fix this, we need to use browser specific Options instead.

From this:-

To this:-

I had to replace all instances in my code that referred to the previous DesiredCapabilities. Once this happened and I saved the file, everything went red. 🤯🤯🤯

I panicked. I thought I’d broken EVERYTHING. I sought advice from a more experienced automation guru than me at this point, and the fix was simple.

To fix this, you simply needed to run a command to update the Maven Project. In Eclipse, the keyboard shortcut for this is Alt+F5 (although I’m guessing the cmd line “mvn clean compile” does the same thing.)

Hurrah! That was all it took! Selenium 4.0 is now enabled 😊.

I’m sure these instructions won’t be applicable to everyone, but if they have helped you at all then that’s cool.

T’ra for now!

What I learned from Mabl’s State of Testing in DevOps report 2021

Here we go again.

A few months back, I blogged about a testing report which aims to cast light on the state of testing. That blog was weirdly popular. So much so that when I saw Mabl had released a similar report, this time focussing on DevOps in relation to testing, I thought it would be remiss of me not to talk about it (love that phrase!).

So, what is the Mabl report all about? This is the third year running that the popular automation tool has surveyed those in and around the QA profession about how, or indeed if, they work with DevOps. 600 people were surveyed, and there are some great insights. You can find the detail here, curtesy of Software Testing Weekly.

Now, if I’m letting my imposter syndrome hat drive my pre-report thinking, I’d have been expecting it to paint a picture of pretty much everybody being fully DevOps. Think multiple deployments a day, 100% automation at all levels of the stack, perfect pipelines and testers (if they even exist anymore right?) scratching their heads thinking “what can we do now?”.

However, as with so many things, the reality lives down to the hype.

Some of the facts that struck me were:-

  • only 11% of those surveyed classed themselves as “fully DevOps & Automated”. The majority were Aspiring, or Striding (in order words, not there yet).
  • there was a 14% decline year-on-year with those who said they had adopted “CI” (continuous integration) principles. 33% said they were in the process of “Transitioning” to utilising CI.
  • Just 27% of respondents said they had adopted Continuous Deployment.
  • How about regular deployments? Almost half of teams still deploy new code less than once a month – so if you’re in a team where that is happening trust me you are not alone. Just 15% deploy daily or more, and this %age has actually decreased year-on-year.
  • When it comes to what layer of the stack to automate, unit, UI and regression are the firm favourites – but all of those have only around 40% of teams saying they automate them.

Perhaps the key one for me was this.

One very interesting revelation from this year’s data is that most teams – regardless of where they are in the DevOps maturity – rely on a combined automated and manual testing approach.

Mabl, State of Testing in DevOps Report 2021, page 17.

We often here “manual testing is dead” type themes from those who want to emphasize the silver bullet type benefits that automation can supposedly bring. I’m with Melissa Fisher here, click this to listen to her insightful Racket on this topic. Automation is a useful tool that can help testers, but it can’t replace them, and nor should it try to. Speed should not come at the expense of quality.

Whilst the companies who are nailing DevOps are more likely to have a culture of quality, and even though they are smaller in number than we might have realized, they will undoubtedly be destination companies for testers keen to embrace future patterns of working. Customers of these companies seem to be happier too, which is a great measure of success in my book.

So DevOps does appear to be on the horizon for a lot of testers. We’re just not all there yet.

It won’t be an overnight move, and if you happen to work somewhere which hasn’t fully embraced DevOps ways of working, whether that’s due to cost, politics or a plethora of other reasons, you don’t need to consider yourself over the hill and unemployable. If you’re somewhere that does bits of DevOps, but is still trying to slowly do more, you’re not alone.

And if you think you have to only do 100% automation to survive in the world of testing, you might want to think again.

Cover image: Online illustrations by Storyset

What I learned from Practitest’s State of Testing Report 2021

Look at the rules, not the exceptions

Now in its 8th year, the SOT Report provides testers with some valuable trend-based information on all things testing. The full report in all of its chart based glory was delivered today, and is well worth bookmarking. You can read the full report here.

Sometimes, when attending meetups, conferences or reading online articles, the loudest voices are often those with the most exceptional experiences. But we may discover this fact long after our own Imposter Syndrome has reprimanded us for not living up to their ideals. We may find ourselves wondering “I’ll never be a proper tester, I don’t even write unit tests” or “I’ve never done test coaching/worked on IoT technology/done BDD/shifted left/[insert plethora of missing skills]” so is there even a future for me in this industry?

Stats tend to be more accurate at revealing general trends. As I did at my Testbash Manchester talk, rather than focusing on the exceptions, I want to pull out some of the rules. How the majority of people who consider themselves to work in Testing define what they do, what they call themselves, and how they work.

Nope, its not sexy. But it is reassuring to learn that out of all the responses:-

  • 28% are known as “Test/QA engineers”, only 0.89% Test Coach and 2.14% are SDETS.
  • 74% and 60% test Web and Mobile, only 9% IoT and 18% Big Data.
  • 92% work in Agile environments, only 27% use BDD.
  • 75% have tasks that involve Test Automation & Scripting, 68% validating users stories. Just 15% are tasked with Unit Testing (and that’s down 3% on the year before).
  • 75% use bug tracking tools like Jira, only 14% use exploratory note taking tools.

Its important to note I don’t want to denigrate the activities on the right. At all. But the fact is, waaaaaaay more folk are doing the things on the left. You just don’t hear about it.

For me, this report puts into sharp relief some of the misconceptions doing the rounds, that you have to know how to do all this stuff to be successful. You don’t. Why? Because as with so much in testing, the answer is often “it depends”. BDD is a great fit for some organizations, but not for all.

Learn the fundamentals, safe in the knowledge that what you may be learning at your current place in terms of processes, tooling or skills could go out of the window at your next company. And that’s totally fine.

Testing is a broad church, and there is still a seat for everyone.

How RuPaul’s Drag Race Has Taught Me to Be a Better Tester – and Why You Should Watch it too!

OK. So I have only just discovered that the new season of reality TV show RuPaul’s Drag Race will air on Netflix Saturday 2nd Jan 2021 in the UK. I cannot tell you how ludicrously excited I am by this news. 🥳

If you’re not familiar with the show, it is a competition with a $100,000 prize fund, where Drag Queens from all over the world compete to win “the race”. It is incredibly popular, and I have loved watching it for years.

It seems a tenuous link, but I honestly think my all-time favourite TV programme has a lot to teach us all. Here are a few lessons I’ve learnt from the show:-

The power of community

If anyone is looking to join, build or grow a community this year, Drag Race is the perfect example of how to do it. It respects its roots, celebrating what came before it such as the iconic “Paris is Burning” documentary and the Stonewall uprising. It also embraces and celebrates all that is weird, “other”, non-conformist and challenging. As testers (and as people), we need to embrace other points of view, and be a part of a psychologically safe workplace. It might sound wooly but it makes the difference between dreading coming into work or staying around for years and years. RPDG provides a space for everyone spectacularly if, like me, you’re into the Testing Peers concept of culture-add not culture fit.

The importance of being authentic

I’ve always loved that Drag Race Queens are selected because of their “charisma, uniqueness, nerve and talent”. A drag act might be all about image and looks, but that alone won’t get them a ticket to the final. What matters is what you do and who you are. Queens who don’t fit the looks mold but excel comedically or are super kind or super talented do spectacularly well.

“Club Kid” looks celebrate the unusual and unique

I’ve always liked that I work in an industry where I’m not judged on my looks. That I come to work most days with no make up on, my hair tied in a crappy bun wearing an old jumper and jeans and that makes zero difference to people’s respect for me or their view on how well I can do my job. Trying to hide who you are at work to fit in is exhausting, and as a long term strategy will likely fail for you. Be yourself, and support others who want to be authentic too.

You will fail. Success comes to those who keep trying.

RPDG has become a phenomenon. It has spawned so many successful global careers for the contestants (who win or lose, yes you Porkchop) that they are often willing to come back and try again. Many who failed to snatch the crown first time around try harder, work on their failures and return to take the “All Stars” title – another RPDG series which takes past contestants and puts them through their paces once again. A lot of the contestants have endured genuine and profound hardships to be who they are, and to even make it on to the show in the first place is rightly considered a major life win.

The power of resilience and persistence in testers is a mighty one too. RuPaul himself says “what other people think of you is none of your Goddamn business” – and I believe this to be 100% true. Do not waste your time stressing about what others think, or being afraid to be the one who says “I don’t understand this”. Push those boundaries Queen.

Shea Coulee, runner up then crowned winner in 2020.

Don’t take yourself too seriously

From the pun-tastic names (“Bagga Chips”, “Heidi n Closet” and “Lattrice Royale”) to the tasks and even the guest judges, everything about RPDG has its tongue firmly in its cheek. Be prepared to work hard, but know life is for living and enjoying yourself.

I always work best in teams with a healthy approach to their work life balance and who enjoy lifting those around them to be positive and have a good experience at work. If your only focus is on delivery on time at all costs, this will lead to burnout, low morale and all the best people leaving. I’ve said it before and I’ll say it again, just don’t be a dick.

I intend to add to this post in future once I’ve watched the latest season (and probably gone back and watched the previous ones for like the 10th time too) but after the year we’ve had I would wholeheartedly encourage you to jump in to the the craziness and life-affirming goodness that is Drag Race.

Can I get an amen? ;o)

Creating A Test Automation Portfolio Bonus Edition: Codeless UI automation using TestProject

I recently completed a marathon project to create 5 working test automation frameworks from scratch, which form a Test Automation Portfolio.  This article is a bonus edition of that portfolio, where I use TestProject to create the same tests in an earlier framework using their codeless solution. (hint: it was insanely easy).

TLDR:

  1. You should consider writing a test automation portfolio if:-
    1. You think you might need to show off your automation skills e.g. to an interviewer
    2. You want to learn more about automation by getting your hands dirty
    3. You want to improve your visibility as a tester and share your knowledge with others
  2. You should consider using TestProject’s codeless test automation for your Test Automation Portfolio if:-
    1. You want a quick (like, lightning quick) way to create tests
    2. You’ve been frustrated by how long it takes to get a fully fledged automation framework set up in the past and want an easier solution
    3. You want to see how far IDE type solutions have come since “the bad old days”

The Project

I had created a test framework which executed a few basic tests such as login on the OpenCart admin demo website – this is a great one to use for testing because it has a lot of features.

I wanted to wrap these tests up into a suite so I could run them all together.

Winning Features

Test Project is the first codeless solution I have used since early Selenium IDE, and I’ve got to say that codeless solutions have definitely come a long way since then! Test Project feels much more solid to me, and yet super easy to get started and have something up and running in no time at all.

The test recorder is a great feature if you want a quick solution.  For example, in an earlier framework I’d spent hours trying to get my head around relative locators in Selenium 4 to identify a tricky logout button that didn’t have a nice user friendly ID in the DOM. I wanted the code to be easily readable, so didn’t want to map some crazy CSS query, and I really struggled to identify the element.

Here it is:-

With the Test Project recorder, all I had to do was click “logout” and it added a user readable step into my test case, no manual intervention required. To someone like me, this is literally hours saved.

Not only did I use the recorder to create all 3 of my test cases, but I also tried out the new sharing feature to pass my test case from one user to the next.  This felt like Postman to me, being able to easily send someone else my code and for them to accept it and run it right .  

You can read more about this feature here.

I was even inspired to do a few video tutorials! Here is how easy it is to create a test:-

And here is a video about how to create a “run” consisting of several tests:-

The code

The codeless solution has some great features that genuinely surprised me.  Once you have written a test, such as login, you can add that test into another test, to avoid re-writing code that you need to maintain in different places.  

Also the use of parameters such as Username and Password was fantastic.  You can set project (global) or test specific variables and even make them secret.  The great benefit of this is if a parameter changes or you want to use a different user you only need to go to one place to update and it will propagate through all your tests.

Thanks to the sharing feature, I thought it would be helpful to share the links to my tests with you. Feel free to import them into your solution and see if it works!

Login:- This test clears the default username and password, types in a new one and logs in.  It then validates the user has landed on the correct page.

View Orders: – This test logs in (reusing the login test above to reduce duplicate code), opens the menu bar and selects sales -> orders.  It then validates the orders page is visible.

Logout:- This test logs in, clicks logout and validates the user is returned to the login page

Lessons learned

Getting the chance to add Test Project into my portfolio was an enlightening experience for me.  I learned how easy it is to over engineer an automated framework, particularly if you only want a quick and easy solution.  The speed with which I was able to get up and running and produce parameterised tests which could run on any browser installed on my machine was a game changer.

Summary

I would definitely turn to codeless solutions in future as the fastest way of getting off the ground, and the fact Test Project has some many features to increase maintainability, such as global parameters, test imports and the ability to share test cases with other team members, means you don’t have to worry that you’re somehow “not doing it properly”.  I also love how you can convert the tests into coded tests using the SDK if you want a more fully fledged solution, as this allows you to start off small, and as your skills mature the framework can grow with you.  If you are interested in creating a test automation portfolio for yourself, I’d encourage you to take a look at Angie Jones article here for inspiration.