Featured

Postman’s Postbot: AIs for your APIs

Hello! I recently wrote a post about 5 Ways Postman’s Postbot AI Can Help Your Testing, and as we know, the world of AI moves fast. So fast, that Postman’s Postbot has spread its electrical wings and flown to other area’s of the Postman platform.

Want to understand the latest release (November 2023) and how Postbot can help you anywhere in Postman? Below I’ve listed all five of the areas which AI is baked into Postman currently, including where to find them and what features you can get from each of them. For giggles I’ve also given them a name, cos that’s just the sort of girl I am 😁 #anthropomorphismFTW

  1. Workspace/General: DocuBot
    1. Where does it live?
    2. What Can It Do?
  2. Collections/Generate Tests: Testbot
    1. Where Does It Live?
    2. What Can It Do?
  3. Tests Tab for Individual Request : PromptBot
    1. Where Does It Live?
    2. What Can It Do?
  4. Flows/Evaluate Block: CodeBot
    1. Where does it live?
    2. What Can It Do?
  5. Generate Documentation: DocuBot
  6. Summary

answerBot 🧾

Workspace/General

Where does it live?

You can find Postbot anywhere in your workspace by clicking the Postbot icon

in the bottom right hand of the Postman app.

Screenshot showing the Postbot button and Prompt pop up in the Postman App

What Can It Do?

This is where you can use Postbot as an assistant, to answer questions from the Postman documentation without you having to leave and go elsewhere.

Some example prompts you might want to enter here could be:-

  1. Ask for help on a Postman feature: e.g. Can you tell me how Postman Flows works?

2. Ask it a follow up question : e.g. Do you have some example Flows I can see?

Testbot 🧪

Collections/Generate Tests

Where Does It Live?

You can find it by clicking the three dot menu next to the collection name and selecting “Generate Tests”. Note this option isn’t currently available from sub-collection folders, its either collection level or individual test level only.

Screenshot showing where to navigate to to Generate Tests for your collection using Postbot

What Can It Do?

You can use the “Generate Tests” feature to quickly spin up test cases for every endpoint in your collection. Here are some tips that might help:-

Promptbot ✅

Tests Tab for Individual Request

Where Does It Live?

You’ll find Postbot where the snippets live, within the Tests tab for an individual request.

What Can It Do?

This feature is a little more developed than the others, and provides some pre-set prompts (think of them just like the snippets) to get you started quickly. Note because the Postbot uses NLP, you can always select a prompt, then tweak using customised follow up messages to get the answer you’re looking for.

Several features are available, including:-

  • Auto-generate tests
  • Visualise the response (e.g. great a graph or chart to visually show your response data in a more digestible format)
  • Save a field from a response (e.g. quickly set an environment variable from response body data)
  • Fix the tests you have (maybe the code isn’t quite right but you can’t figure out why)
  • Add more tests (bear in mind the tips above – don’t assume this is going to automatically give you what you need, but as an inspiration provider it could be useful).

Codebot 👨🏽‍💻

Flows/Evaluate Block

Where does it live?

You can find the Postbot for Flows by selecting an Evaluate block and adding it to a Flow. When you click the purple Flows icon, you can enter your prompt.

Screenshot showing where Postbot lives in Postman Flows

What Can It Do?

This Postbot can help you with the coding side of things. Instead of learning Flows Query Language, you can try prompting Postbot to write an evaluation query for you. Moving you one step closer to no/low code implementation, even for more complex queries.

In this example, I’m using the Mailinator API to retrieve an email inbox, and count the number of messages it contains:-

Screenshot showing Postbot in action in Postman Flows

Docubot 📃

Documentation Tab

Where does it live?

You can find the Postbot for Documentation by clicking Documents at the right hand side of an API endpoint.

Video Courtesy of Abhijit Kane @ Postman

What Can It Do?

This Postbot can take the task that so many API developers struggle to complete, one of the most common complaints people have when trying to consume an API: Documentation. It can read your API and generate useful documentation – or at least a draft which has done a lot of the heavy lifting that you can review and make tweaks to. A lot easier than starting from scratch, no?

Take a look at Postman’s latest blog post to find out more: https://blog.postman.com/docs-with-postbot/.

Summary

Postman are working AI into their platform at pace – I suspect this blog post itself will be out of date possibly within a few weeks. But currently, there are 5 different places you can go to within Postman (desktop and browser apps, Postbot isn’t in the VSCode extension, yet!). All of these implementations can be thought of as different bots, to help with different tasks.

Hope this helps you. And just a reminder, I don’t work for Postman, and I don’t get paid in any way shape or form to give you this information. I figure its a useful mental shortcut I can use in future in a tool that I like, and if it helps me then with any luck it could help you too!

Featured

5 Ways Postman’s Postbot AI Can Help Your Testing

Update – 15th Sept ’23. Thankyou to the folks at Postman for quoting my blog in their latest article about Postbot.

Screenshot of my quote from this very article taken from Postman’s latest blog post about Postbot #meta

Now then testers, quality engineers and generally inquisitive folks! I’ve had a sneak peak at the new AI bot Postman have been working on in their canary version – you can apply for the beta yourself here if you want to follow along and take a look. In my usual style, I thought I’d drop 5 handy ways you can use it at work, if you use Postman for your testing already. It can’t replace you and your fabulous testing brain, but it can help with formulating your assertions and getting your idea from in your head to a working script.

Postman have dipped their toes into the AI world already with Flows, in a pretty unannounced feature called AI assistant, that you can use to write FQL (Flows Query Language).

YouTube video showing Flows AI Assistant in action

However, now OpenAI integration has made its way into… dah dah dah…. the tests tab! 💻

picture shows “script with Postbot” in snippets section of Postman’s API Platform Test tab

Thats right – the snippets side bar has a new friend 🫂

TLDR: Remember when you first discovered snippets and your jaw kinda dropped at how much time it was going to save you? You ain’t seen nothing yet.🔥

From my first thoughts, this could quickly become a habitual way of writing tests in Postman. It is very easy to use, and generally speaking, very good at coming up with the right syntax. Most (90% or so, depending on complexity) of my Postbot generated code was what I wanted, and worked first time. Gamechanger territory my friends, especially to those of us to whom code takes time to write, with a lot of Googling and tweaking involved.

1. Ask Postbot to write tests for you

Youtube video showing Postbot generating some tests from a prompt

Using the prompts within Postbot, you can ask it to auto-generate tests for you. You won’t know the logic behind them, but you can see if they generate you anything useful, and get rid of the trash. In this instance, I sent the GET/Pokemon API, then clicked Postbot’s prompt. Several tests were generated, including the old faithful status code check, alongside more specific checks for relevant content. A few things that I like:-

  • It nests assertions within the same test, rather than a separate test for each
  • It labels each test with a useful name – something that I often forget to do in copy/paste land to my detriment come debug time!
  • It checks security as well as functional areas (content of array responses)

A few words of caution though – when playing with this I did notice that sometimes incorrect syntax was generated (e.g. if you already had a test and the cursor was placed inside it when Postbot was asked a question). It also ignores any existing tests – so if you already have status code == 200 checks then be prepared for some dupes!

2. Ask Postbot to write you a custom test

There will always be cases where the AI isn’t intelligent enough to ready your mind and come up with the exact scenario you were thinking of. So before you start writing the code, why don’t you see if you can ask it for what you want. I was pleasantly surprised by how complex prompts were understood.

3. Tweak a custom test

If you used ChatGPT before the prompts were remembered, you know how painful this was. Luckily, Postbot stores your last request, and allows you to adapt it based on what you need. e.g. From “get the value” to nah, actually, “get the value in the first array response”. or actually, I wanted it to ignore the casing of the environment variable, that’s giving me a false negative”. Just open up the prompt again, tweak it and it updates the test – nice!

So go from this…

Screenshot shows original prompt, test code and false negative test result

To this…

Screenshot shows tweaked prompt, updated test code and positive test result

Again, a note of caution. Sometimes Postbot tries to be extra helpful when tweaking, and inserts brand new tests that you haven’t actually asked for. Sure this will improve over time, but be mindful of the before and after.

4. Ask Postbot to store response values as variables

You can use the set and get variable snippets for this, but if you want to ask Postbot to do this as part of a test you can.

Screenshot shows Postbot prompt for variable selection and accompanying test

Note – Postbot’s mission in life seems to be around generating tests atm, so if you’d prefer this to be a non-test then use the code snippets instead. Handy add-in to tweak for an ai generated test if testing workflows for example.

5. Visualize Response Data

So I’ve pretty much given up on visualizer as a feature, because writing the code to configure it was so tricky. However, the auto prompts in Postbot work a treat.

In this example, I use the Mailinator Stats API, and ask Postbot to generate a bar graph of the results. Et voila!

Screenshot shows Postbot prompt, auto-generated code and visalized bar chart of response data

When I tried to create the prompt myself, the code didn’t work properly, but Postman seem to have trained Postbot to come up with the goods when the prompts are selected e.g. Visualise Respose… as a table. Then select the body tab, click Visualize button. This could be genuinely helpful.

To sum it all up…

Of course, treat this and all other AI with caution, review what it tells you and make sure you’re not overdoing it #sustainability. Finally, using a feature like this means it is more important than ever to TEST THE TESTS! Before you blindly rely on them, make them intentionally fail, so you know if you need them to in future they will.

Even in its infancy, with some finesse to do, Postbot is still a mighty feature. Definitely for new and experienced Postman users alike something worth having in your toolkit. It isn’t a replacement for an experienced tester, who can assess nuance, applicability and sustainability. But its a pretty blinking decent assistant. Good job Postman

Good Things I Read or Saw This Week 07.01.2024

I’ve heard recently that a lot of folks use blog posts as a bit of a memory aid 🤔 🧠

They link to stuff they’ve read or perhaps want to read later while its fresh in their mind so they can go back and look later down the line.

I’m going to try this out with a few of the best things I’ve read or seen this week (week ending 7th Jan 2024). I’ve included links to everything so I can go back and check it out later, but feel free to take a look yourself if you’re reading this – it was mostly written for me but that’s no reason you can’t benefit from it too lovely reader!

API Masters Platform

APIDays have created a new free learning site called ApiMasters – https://www.home.apimasters.io/. think of it like a Test Automation University for APIs. It already has several free, non-sponsored courses in there from industry professionals that I’ve managed to complete, and I hope to see more development here in the coming months. Highly recommend for anyone who wants to learn APIs more broadly e.g. API product management, API Security, API Documentation ⭐⭐⭐⭐⭐

Screenshot showing the difference courses currently on offer at API Masters

Designing APIs that stand the test of time

Speaking of APIs, I’ve been checking out Pooja Mistry‘s talk she did at API Days Interface. I like seeing talks aimed at API developers and designers, because it feels like you get to see how the sausage is made. It actually helps quality engineers a lot, because you can see the typical issues that a lot of a developers have when crafting APIs, and examples of what good looks like so you can refer to these when testing APIs of your own. I was lucky enough to meet Pooja in person at Agile Days Chicago last year, and she’s just an incredibly cool person – anyone who can explain techy concepts to non-techy folks always gets my vote!

Things Wot I Have Asked ChatGPT this week

I bought myself a chatGPT subscription for Christmas (don’t laugh!) and am still using it, but not necessarily for the high-fallutin’ reasons I originally thought I would!

It actually reminds me of an earlier experience I had. I once contracted for a small start up, who were making really cool home security cameras. They were bought out by a bigger company whilst I was there, and the bigger company made a really good point about marketing. They said:-

There’s a huge difference between people’s buying intention and what they actually use something for. People buy indoor cameras because they care about home security, and *think* they want to use them to prevent intruders and very scary situations from happening. But the product is actually rarely used for that purpose. In fact, 99% of the time the camera is used to take cute photo’s of the dog.

And I kinda think that chatGPT is becoming like this for me, at least until I find a good use for it. I’m glad its there when I need to ask it something incredibly high brow and stuff, but in the cold light of day what I’ve actually been using it for is to get to what I need without wading through Cookie notifications (that I always decline so can get painful), Ads or content that I don’t need. For this it is incredibly useful. For example:

  • Give me a 15 question quiz with answers on the first three Harry Potter books (these are all that my little boy has read so far and he hasn’t yet seen the movies so getting a specific set of questions without the appalling sign up on the Harry Potter website is great!)
  • Summarise how to solve a Rubik’s cube
  • Does UK accent bias exist? I’m very interested in this topic, having seen something on TV about it a few months back. I’m toying with the idea of doing a talk, but wanted to do some research first. (spoiler alert, it does exist).
  • Dall-e – generate an image of microscopic close up art of an eyelash
Picture shows Dall-e AI generated image in black and white of a eyelashes complete with skin mites – eurgh!
  • Identify 5 trends in software testing in 2024, show your sources

All of this was done from my phone, mostly while I was a passenger in a car, or watching TV. I also experimented with Bing and Bard, to see if they gave similar answers to the same question – definitely worth knowing this and trying alternatives if your AI isn’t giving you what you need!

A Realistic World View on AI adoption: Getting AI Ready in 2024

Great blog post from the Ferrio team.

Screenshot shows a linkedin Post of an article entitled “Getting AI ready in 2024”

Software Testing Weekly – issue 204

I subscribe to Dawid Dylowicz fantastic testing roundup, so when an email lands in my inbox I immediately check out two or three of the articles that stick out to me. This week it was a Reddit thread on QA skills learned – I don’t usually view Reddit but the posts on there are usually the ones that feel most honest and uncensored, so its a useful prompt for me to get links to good threads.

Ministry of Testing

I usually check out a couple of things on MoT every week, because its such a trusted source of information to me. I enjoy Friday afternoon live LinkedIn sessions with Simon Tomes (community Boss) as anyone can jump in, and so you feel like sometimes the unstructured convo’s that bubble up on there can really get you thinking. Ditto Club posts. I also read Ady Stokes article about Making Your Presentations More Accessible. Keeping on top of accessibility is super important, and next time I have a new presentation I’ll come back to this article to double check the accessibility considerations I’ve made are good ones.

Summary

And that’s it for this week – hope you re-read this in the future Beth!

2023 Last Minute Gifts For Software Testers

Finally getting round to Christmas shopping? Realising you haven’t got time to order something and get it delivered in time for the big day? No probs!

Here are a few ideas for thoughtful techy gifts that can be bought and shared instantly – no p&p or next day delivery charge in sight whoop! 🎁

Here are my top, and 100% unsponsored, tips:-

Books

There have been several stellar software testing books released this year. Available digitally (hello last minute.com) or in good old fashioned paper form.

  • Ai Assisted Testing, MEAP, early release – Mark Winteringham (Dec ‘23).
    Mark’s latest release, a follow up to his excellent book Testing Web APIs (2022) – this hot-off-the press e-book focusses on practical ways to integrate and use AI as a tool to support effective testing. Disclaimer – I have reviewed this book, and can testify its a great read!
  • Software Testing Strategies: A Testing Guide for the 2020s , Pakt, Matt Heusser, Michael Larson (Dec ’23). Looking forward to reading this book. Matt and Michael are very experienced folks with their fingers on the pulse of what is going on in testing and this is sure to be a knowledge-filled and insightful guide.
  • Performance Testing Handbook, Leanpub, Mohamed Tarek (May ’23) – Mohamed is a friend of mine, and I was delighted to write the forward to this book. A really great practical guide to performance testing in your organisation.
  • Finding your Mojovation, Leanpub, Neil Studd (May ’23) – Another book that was one of many who helped review. Neil brings his journalistic fluent writing and research style to bear on this book. Finding your mojovation has lots of incredibly relatable real-world insights from a software tester who has worked in the industry for many years.

Memberships

Often, us testers aren’t fortunate enough to work for an organisation with a huge personal training budget. So we miss out on anything that requires a subscription or paywall, which can make finding decent content a bit more tricky. Want to help?

screenshot from MoT website with reasons to go Pro

Other bits

Of course, most testers just want regular nice things as presents. But if you want to get something more generic, or even give them the choice of something to get that’s still thoughtful, maybe a voucher could do the trick?

  • Home Office Stuff – stuff to make that home environment a bit nicer – think plants, posh stationary, maybe a small whiteboard or a cool picture. Designworks do some great bits.
  • Gift Card – did you know you can get a Ministry of Testing Merch gift card? Now you do! Lots of cool SWAG on there including Testsphere cards, would Heu-Risk-it and all the hoodies and caps a tester could wish for.
  • Postman Store / Red Bubble also have some fun themed t-shirts, hoodies and the like
sample images of Designworks stationary

Please note – I am not sponsored by any of these people or companies, they are just genuinely things I think are good.

Hope it helps put a smile on the face of a QA who deserves it. Feel free to add a comment to post your top prezzie tips.

Happy holidays.

5 Key Takeaways From AgileTDUSA ’23

Speakers photo at AgileTDUSA ’23. I am on the far right, middle row.

At the end of May 2023, I attended, and spoke at, AgileTDUSA – my first US based conference. It was based in Chicago, and it was a real bucket list goal of mine to have achieved. Before I went, I had a few thoughts:-

  • How would the conference compare to others I’ve attended, mostly in the UK
  • What would the experience of speaking on behalf of a sponsor be like?
  • Who would I meet there, and what would be the vibe generally?

So lets cut to the chase and look at my key takeaways – note not all of these are technical, or even about testing specifically, but more a set of mental reminders to my future self so I can recall more easily in future. Also – there will be some serious testing namedropping here, which I make no apologies for!

1. People are people – but testing people are the bestest people

I’ve said it before and I’ll say it again – testers are my tribe. There’s nothing quite like the feeling of walking into a room and knowing you are welcome – and seeing so many people you know or feel like you’ve met in person even when you haven’t! I met too many people to mention here, but they were without exception kind, supportive, funny and interesting, and able to be completely themselves here too.

I just need to bear this in mind when attending future conferences – that if there’s someone there you are really excited to meet, a small token or gift is a really nice touch. Special thanks to Prici Bilavendran and Jigesh Shah for your thoughtfulness.

A tweet showing gifts, swag and goodness from the testing community

2. Mailinator has a bit of a cult following don’t they

I mean, I should have known this already, but it was still very cool to feel like one of the in-crowd wearing a Mailinator branded polo shirt at AgileTDUSA. Honestly, the number of people who’s jaws dropped when they realised that *the* actual CEO of Mailinator was in the building was almost comical. I had a huge smile on my face the whole time, because I don’t support tools that I don’t believe in, and this confirmed what I’d always known, that LOTS of other people love Mailinator and what they do too – truly in a big, fan-girl/boy type way. Just awesome to see.

The Mailinator Team sponsor shoutout at AgileTDUSA

Getting to spend time with the rest of the team (who get a bit shy so I won’t name them here) was also so so nice. The icing on the cake was all going out for a meal the night before the conference. Pineapple-infused vodka FTW.

3. Digital notebooks are useful AF and I should definitely get one

So, it was an absolute delight to meet the one and only Tara Walton (who’s by the way open to work, hire her immediately). Among the many chats we had, I noticed her Rocketbook and she was kind enough to let me get my hands on it. I know I’m a bit behind the curve on this one, but I think this kind of digital notebook would really help me with point 5 below.

4. Apply Dan Ashby’s model to my talks

I know I didn’t say so at the time, but I will always remember and genuinely appreciate those who took the time to attend my talk for Mailinator. Yes especially you Vernon Richards – hope you enjoyed the SWAG we gave away!

Lets face it, a sponsor talk is never going to be the headline, and I had some amazing speakers giving talks and workshops same time as me so I wasn’t feeling too confident. But seeing friendly faces that I knew, sitting in the front row, who were happy to get involved and wouldn’t stop smiling at me (you know who you are!) was a huge confidence boost. So much so that for the first time I actually felt able to go off the cuff a bit, and, dare I say it, even make people laugh!

To hijack Dan Ashby’s quality analogy, (demonstrated here by the very experienced and very gifted speaker João Proença, also at AgileTDUSA) I struggle to inject comedy into my talks because I get too obsessed with them being <technically> “correct” and “useful”<for the audience> that I don’t think enough about the “goodness” <e.g. is it a fun talk that leaves you with a smile on your face>

certainly a reflection I’ll take with me to AutomationSTAR in Berlin in November!

Tweet from EuroSTAR Conferences about the workshop I’ll be giving at AutomationSTAR

5. I wish the talks were recorded and should have done more to remember them

This sounds like a huge miss on my part – and it was. I am so used to conferences recording the talks that I didn’t mentally bother to make enough notes. I wanted to actively listen during the talk, do my bit as an engaged attendee and not spend the whole time looking at a notepad – but now I realise that without the recordings to fall back on, there are a heck of a lot of blank spaces where good information should have been. Some ways of minimising this:

  • Having access to the slide decks – thanks organisers
  • Favouriting the amazing Lisi Hocke’s set of sketchnotes – she did the hard (glue) work and I can benefit from her excellent insights in this really digestible way – thanks Lisi! As an aside Lisi, João and I briefly worked together last year and it was really lovely to meet them both in person at this event. Just all round great people to be around.
  • Using the limited notes I took and at least making this token effort to write them down in some way – see future blog post

For future reference though, I should add this to my mental checklist of things to run through before attending future conferences, along with bookmarking the talks I don’t want to miss and giving myself alarms to make sure I definitely attend them and don’t get carried away having exciting conversations with Lisa Crispin or Janet Gregory.

All in all, my first international conference was a big hit with me. Amazing organisation, brilliant speakers, plenty of that good good swag, and just a chance to relfect on the industries past present and future.

TTFN

A Software Testers Love Letter to QVC: What The Shopping Channel Can Teach Us About Quality

N.B. I am not sponsored or paid to write content about any company, product or feature. I simply write about things that I genuinely love.

For many many years now I’ve loved the shopping channel QVC. And I’m with Fenton Bailey here, I refuse to be apologetic about it. They have been going in the UK for 30 years, far longer than most technology companies and certainly longer than most TV programmes.

I don’t actually enjoy physically going round the shops much, but there is something about the psychology of watching people try to sell stuff to other people in a non-invasive way that fascinates me. And as an avid viewer for years I can attest that QVC are very, very good at what they do. TV aside, they are also spectacularly tech savvy, know their customers like the back of their hand, and can offer software testers and the wider tech community some nuggets of wisdom, provided we are open minded enough to take them seriously.

In this post, I’m going to advocate for being “a bit more QVC” in a few key areas. Personally, I struggle to resonate with similar posts drawing parallels with sports, historical figures or super tech heavy things, so I hope there are others who share this view and find the following a bit more sticky.

The Power Of Personalities

QVC very rarely, if ever rely on a single presenter. Go and put the screen on now, and you are likely to see at least two people. Usually, a QVC key presenters such as a general fashion expert or beauty officionado, alongside someone representing the product.

Screenshot of QVC TV, showing two presenters demo-ing a product

And it works. It comes off as more persuasive and more trustworthy watching a conversation rather than a lecture – I guess its for the same reason that a lot of sports have both the “technical” and the “colour” commentator.

I wonder if we could be more QVC in our approach to documentation, show-and-tells or just plain conversation. Put our technical, SME hat on to talk about a products key features, key technical observations etc. Then either ourselves, or a co-presenter/pair-tester etc. provides the human commentary – uses humour, empathy and story telling to win folks over.

The Power Of Persuasiveness

Speaking of winning folk over, I often think when watching QVC how well trained the presenters must be to convince thousands of people to do something in the space of just minutes. One sales technique they use to good effect is to give use cases, or examples, of how a product could help solve a problem you have had. Simple anti-virus sticks, RFID protective wallets or even security cameras all talk about the potential of what *might* happen. And mitigating against risk is no different really is it? We could be more QVC by being empathetic in how we communicate, thinking about what is going to resonate with the “audience” or stakeholder, and giving them real world examples of what could happen to help to persuade them. And do the same in our talks – make it about whats-in-it-for-the-viewer, not how-much-can-the-speaker-show-off-what-they-know.

The Power Of The Sincere Apology

Issues are addressed immediately – because believe it or not, you can’t lie on telly, so if someone makes a claim that isn’t technically true (e.g. provide clinical studies or industry recognised evidence to back up what they are saying) there are financial implications as well as trust ones. So any issue, no matter how slight e.g. “the measurements we read for that top don’t look right” or a discrepancy between what colour a chair is called on the website vs. on air, is called out, apologised for, and rectified.

I recently had an issue with an unnamed tech company, who mid-way through a bug they had caused by a new release which froze accounts actually went so far as to remove the customer support page where folks were reporting the problem! This kind of opacity drives me bonkers, and I think there is definitely a lesson to be learned there about proactive and diligent comms, and a culture that expects the odd mistakes and provides a human way of dealing with them. Be more QVC.

The Power of Embracing Change

QVC adapt. Constantly. They are masters of reviewing what works, adapting their model and unapologetically binning off product lines that their loyal customer base no longer resonate with. Perhaps we should be a bit more QVC when maintaining our (automated) test suites?

When it comes to the tech, QVC are clearly using it wisely. No access method to purchase something is off the table. From on screen QR codes, “tap the app”, widget, using the website or the good old speak-to-a-person-on-the-phone, its a reminder to continually truly understand your customer base and create your ecosystem around their range of wants – not just the single method that’s easiest and cheapest to implement for the most people.

Extract from QVC UK website, showing the different social media and native apps they support

The Power Of Customer Feedback

QVC’s unflinching, relentless focus on customers, allowing them unfettered and unfiltered feedback on all products they sell, is a way we could be more QVC. Perhaps the ultimate twist is Qurio – a Social Shopping App – aka free user research!

Extract from QVC website introducing Qurio social shopping app

QVC are so confident in what they do, that they use an app that lets their customers give personal video reviews of their products. And what’s more – their customers do not get paid to do this, the community is so strong that people will spread the word about the products they buy for absolutely nothing. I wonder how many companies would love that sort of insight. If we wrote, tested and released software with a view that each and every single customer of that software had a direct and publicly visible way to feedback how much they liked it (or didn’t like it) – be honest, how much would this force us to change our whole teams view on quality? How many more conversations would we have? How much more adaptive would we be?

Be More QVC

Clearly, I am an unapologetic fan of QVC. I’m obsessed with their human centred approach to everything they do, and look forward to continuing to be inspired by their work. QVC even have an accessibility brand ambassador, a series of shows and bags of content focussed on menopause, and hundreds of podcasts to add depth to their content.

To paraphrase one of their own tag lines, if you think you know QVC, and think it can’t teach you anything about tech, you might want to think again.

5 Reasons You Should Consider Creating A Software Testing Portfolio

As you might know, I’m a huge fan of Test Portfolios. As a quality engineer who found their voice during the Covid pandemic, I’m living proof of the importance of having a software testing portfolio. In today’s fast-paced and competitive tech industry, having a portfolio can be the key to landing your dream job, increasing your confidence, and inspiring others. In this blog post, I’ll be discussing five reasons why you should consider creating a software testing portfolio.

What is a Software Testing Portfolio?

Slide showing my definition of a portfolio, alongside a cartoon explaining the concept of Narrative Bias

This is my own definition, there isn’t really a commonly held one. For me, the things that make a Portfolio the next level up from a CV is 3 things, namely:-

  • They are Dynamic – they can evolve over time, and contain living, working code extracts etc.
  • Narrative bias – a portfolio is easy to wrap a personal story around, to make you the person who is front-of-mind for the people you want to get noticed by
  • Depth – They can be as shallow or deep as you want, but often CV’s don’t give the detail that really sells that you know what you are doing – a portfolio is your way of proving you aren’t just playing buzzword bingo

Now, lets take a look at the 5 main reasons I think Portfolio’s could give you a leg up.

1. Landing a Job in the Hidden Jobs Market

Did you know that there is a hidden job market out there? This market consists of jobs that are not publicly advertised, and are instead filled through networking and referrals. According to a report by Forbes, up to 80% of jobs are filled through the hidden job market.

This means that if you want to land your dream job, or sometimes even just your next job, you may need to tap into this hidden market. And one way to get noticed above the other candidates is by creating a software testing portfolio. A portfolio is a great way to showcase your skills and expertise to potential employers. It allows you to demonstrate your testing methodologies, strategies, and results in a clear and concise manner.

At the Coders Guild, all aspiring Quality Engineers are mandated to create a portfolio, and I can testify that in showing them off on LI profiles, covering letters and at interviews, they have been the difference in landing them their first roles in QA.

When you have a well-crafted portfolio, you can use it to approach potential employers and hiring managers. You can share your portfolio with them during interviews or networking events, and they can see firsthand the value that you can bring to their organization. This can set you apart from other candidates who may not have a portfolio. Bonus points if you do something with the website or tech that the place you’re interviewing at uses.

2. Increasing Confidence (You *can* do it!)

One of the benefits of creating a software testing portfolio is the increased confidence that comes with it. That’s basically why I did mine. I was sick of feeling “not technical enough” and wanted to prove that I could learn this stuff with some application and graft. When you have a portfolio, you have tangible evidence of your skills and accomplishments. You can see the results of your hard work, and you can use that to build your confidence. People also love helping you on your journey, and I was amazed that every single tutor, industry professional and community champion I asked was more than happy to help me get over those inevitable bumps in the road.

We know that the only constant in tech is change. Creating a portfolio forces you to reflect on your work and think critically about your testing methodologies. This can help you identify areas for improvement and can help you grow as a quality engineer. As you continue to add to your portfolio, you’ll see your skills and expertise grow, which can further boost your confidence.

3. Code Reuse

Creating a software testing portfolio can also help you with code reuse. One example, I wrote a short Selenium automation framework by following a Test Automation University course (all free!). Mailinator saw the blog post I did alongside it, and asked if I’d like to write a tutorial on how to use Selenium with their email testing platform. Because the code was already written, it was easy to adapt it to the new site. And it felt great to get further value from something I’d already completed.

I don’t advocate copying code from work (of course!) but if you have learned a new technique or concept, why not write some boilerplate code you can reuse in future? These scripts and code can be reused for future projects, saving you time and effort.

4. Side Hustle: Opportunities to Speak, Write and get involved

Creating a software testing portfolio can also open up speaking opportunities for you. When you have a portfolio, you have a wealth of knowledge and experience that you can share with others. This can make you an attractive speaker for industry events, conferences, and webinars. I’ve spoken about Portfolio’s at international conferences, keynoted at Ministry of Testing events alongside writing for Teatime with Testers and many others. In fact, my Test Automation University course came about because Angie Jones read my portfolio blog (which she inspired!) and saw I had an interest in Postman.

Speaking at events is a great way to build your personal brand and establish yourself as a thought leader in the industry. It can also help you grow your network and meet other professionals who share your passion for software testing. And let’s face it, it’s always fun to share your knowledge and experience with others.

5. Inspiring Others

Finally, creating a software testing portfolio can be an inspiration to others. When you create a portfolio, you’re showcasing your skills and expertise to the world. This can inspire others who are just starting out in the industry, or who are looking to take their skills to the next level. I have been amazed at the number of people who have reached out to me to tell me about their portfolio journey.

To Sum It All Up

A software testing portfolio can bring many benefits to a quality engineer. It can help you tap into the hidden job market, increase your confidence, promote code reuse, open up speaking opportunities, and inspire others. Creating a portfolio is a great way to showcase your skills and expertise to the world, and can help you stand out in a competitive industry. So why not start creating your own portfolio today? It’s a worthwhile investment in your career and personal growth as a quality engineer.

Take a look at my portfolio here if you want some inspiration – it has all the accompanying blog posts with it along with the github links if you want to delve deeper.

Enjoy!

ChatGPT for Bespoke Test Data Generation

I’m seeing a lot of brilliant posts that I’ve learned a tonne from all about ChatGPT and what it can do to help us Quality Engineers. I’ve also posted about what a contentious topic this is to even acknowledge you are using ChatGPT – my earlier musings on this topic seem to have been borne out in my experience so far.

However, the one thing I haven’t yet heard anyone go into is using ChatGPT to create test data generation.

TLDR: Watch this video to see how I did this

There are a few advantages to this:-

  • Realistic data sets – We know ChatGPT-3 isn’t the latest data set, but it is at least based on a heck of a lot of data. So maybe you want to know what the most popular products are for your company, or what the best grossing films were in 2010 in Morocco, or what the most well known technical trailblazers names were. If its something where you care about what others think, as opposed to having a linear set of something e.g. 1-200, then being able to tap into those data sets could help you be more realistic
  • Bespoke Test Data – there are already brilliant libraries you can use (such as faker.js) which auto-generate test data for you, but what if you need something more specific. One example could be you have a field that requires first names, but you want to only use female first times. That isn’t a sub-selection you can currently do out of the box.
  • Something fun for Demo’s – want to spice up a customer demonstration, or an end-of-sprint show and tell? Plug in unique test data and ask it for something wild!

I spent an evening a few weeks back solving this puzzle.

I created:-

  • an open-source, free, publicly available workspace in Postman
  • Using Postman Flows (the low code workflow builder feature) you simply modify a query that you feed in to ChatGPT using the template provided, and Flows formats the response that comes back to allow you create an array of comma separated test data
  • This test data is then immediately plugged into an API request – showing the end-to-end process of test data generation and looping through a request for each and every bit of test data ChatGPT Provides – so you get to select how many times you want this to run by asking for that number of items in your query.

The results are in this video:-

YouTube Video Walking Through OpenAI Test Data Generator

Pros and Cons

  • ChatGPT will not be free forever, so this may have limited shelf life unless you’re willing to pay for access to the API
  • Asking for large datasets may use your free tokens pretty quickly
  • If you ask the same query, ChatGPT will come back with the same answer. So its really important you know what to ask it if you need randomised data every time the query runs
  • You don’t know how accurate the data is – so be careful what you are asking it for and how much you rely on it as a source of truth

I hope this helps you in some way, as this is quite a novel way to use ChatGPT from what I’ve read (although I’m sure someone will create a more user-friendly tool version that does something similar soon if they haven’t already). As with all of my posts, videos and community work, I didn’t get a penny for creating this or putting it out there, so if you do find it useful please remember to say thankyou and quote your sources, it gives me the impetus to keep going!

T’ra for now 😜

Postman Flows: 1 Flow To Smoke Test Your API Estate

One of the things that I love most about airing a new talk is the new ideas it sparks. On Friday, I delivered my talk “Go with the Flow” for the first time, to the lovely internal community at Dunelm.

LinkedIn Post from Dunelm, who were the first to give feedback on my new talk around Postman Flows

The inspiration for this post goes to Adam Pike of Dunelm, who during the talk mused that Flows would make it possible to see the full suite of APIs interacting with each other in Postman.

And it got me thinking. We know Flows has this superpower of being able to call any other collections in a workspace. If you have different collections for all your different APIs, wouldn’t it be handy to be able to visually see the key connections and check they’re all talking to each other?

Here’s a couple of ways I think it could work.

Simple Ping Smoke Test

Example smoke test Flow showing several requests being checked and errors logged out to Slack

In this example, I’m using 3 requests which can belong to different API’s. I’m then checking for the status indicator – however, you can just as easily check for status code != 200 or in a range of allowed values or similar. This then outputs to the console log if all is well, or, if there is a problem, a Slack message is output to the team with the Flows ID to alert them to an issue.

Advanced Integration Smoke Test

But what if what you want to keep an eye on is whether API’s are working when strung together in a real-world customer journey? Will the Auth API talk to the Orders API? Will the Location API talk to the 3rd party Google API? Perhaps you have mocks for third party services, but want to check if they return their end of the bargain and give you what you expect, that your internal API’s will respond in the right way?

Flows can allow you to create that visual map of your API estate.

I see it developing like a spiderweb, with a start button in the middle branching off into different workflows, all executing at the same time. 🕸️

Here’s a very rough idea of what that could look like:-

Flow containing 2 integration tests running concurrently

The Yellow Flow shows API 1, authenticating, posting and deleting. The Blue flow shows a different API flow entirely – you can of course use multiple APIs in a single flow if you wish. Using the colour feature we can code each flow to make it easy to see if they are running as expected, tagging a different API in a different colour (maybe mocked APIs in grey, for example). If not, we evaluate the status code and if we don’t like what comes back, we send a message to Slack.

Once you are confident you aren’t generating heaps of information that will just be ignored, and no false negatives(remember to test the tests!), you might even want to get this up on a monitor and schedule a run for every 5 minutes or so, just to give you the confidence everything is up and running in the environments you need things to work in.

Thanks again to Adam for the inspiration – I think this might be a useful feature to have.

Hope this helps, til next time!