I defy any responsible, sober technical professional to visit this website and discover what the “MABL” tool is and does without reaching out to the company to beg for actual details. It has an introduction video, for instance, that conveys no information whatsoever about the product. Yes, it is teeming with sentences that definitely contain words. But the words representing irresponsible, hyperbolic summarizing that could be applied, equally irresponsibly and hyperbolically to lots of different tools.
My favorite moments in the video:
0:33 “write tests… just like a really smart QA engineer would.” Huh. I would like to see a QA engineer go on the video and say “I’m really smart QA engineer, and MABL does just what I do.” I would like to interview such a person.
0:44 “She uses machine intelligence to…” Yes, the talking man is using the female pronoun to imply that “MABL” has the tacit knowledge of a female human engineer. Isn’t that nice? He speaks with a straight face and an even tone. He must have a lot of respect for this imaginary woman he is marketing. (Note: no human women speak on the video, but there is one in a non-speaking role for about a half-second.)
Ultimately, I am left not knowing what specific functionalities their tool has that they are lying about. Yes, lying. Because their claims cannot possibly be true, and they cannot possibly believe they are true– kind of like one of those infomercials about 18-year-old girls in your area that would love to talk to you. Except in this case, her name is MABL and she wants to test your product.
What is really going on?
Apparently the industry has reached a point where testing services can be sold the same way miracle weight loss programs or anti-aging face creams (with micro-beads!) are sold. This can only happen in an industry that holds testing craftsmanship in utter contempt. The testing industry is like a failed state ruled by roving gangs.
Maybe this MABL tool does something interesting, but it seems they don’t want us to worry our pretty little heads about it. And that is something that should worry us all.
A review of a promotional video by someone who took absolutely no time to see what the tool actually is capable of. Will it replace testers? No. But it does not claim that.
[James’ Reply: As I pointed out in my post, they LITERALLY claim that.]
Is it an impressive tool? Yes.
I am a “responsible, sober technical professional”, at the cutting edge of Automation, who actually works in the field every day. Unlike the author of this clickbait.
[James’ Reply: There is no information on that website about their fucking tool. And you fucking know it if you have looked at it.
BTW, I don’t know you. I’ve never heard of you. But if you’ve been in the industry or know how to use Google then you’ve heard of me. You ought to know that I stand against bullshitters and you ought to be trying to help me, instead of offering no actual information about this tool you say is so impressive.]
[comment redacted]
[James’ Reply: What is it with you kids? Such weak rhetoric! Look, if you want to criticize what I said, then A) address the point that I was actually making, and B) supply evidence and apply reason. For instance, you say that you do “actual work” but you don’t point to any of it so that I can evaluate it. And why do you say that I don’t do actual work? You don’t actually know what work I do, do you? And if you did know, we would have to discuss the relevance of our contrasting work histories. For instance, I bet we would discover that you have no background in studying or analyzing testing practices, processes, or testers themselves. I suspect we would discover that you are not qualified to give an opinion about the value of a tool to help you test, since you don’t know how to make an argument about what is and is not good testing.
You say I pontificate, yet the act of saying that is a pontification itself, and thus your rhetoric disappears up its own ass (that’s what I meant when I said it was weak).
Since your incoherent babbling does not constitute an argument, in order to reply to you the first thing I would have to do with your comment is to deconstruct it, reconstruct the argument that I guess you would make if you were a competent analyst, and then reply to THAT. That’s too much to ask, man.
I invite you to try again. First, read my post and see what I’m trying to communicate. Then decide whether you want to dispute my way of saying it or the facts I offered or the conclusion I drew. Then offer evidence and reason, or at least an alternative narrative that hangs together.
Another commenter, for instance, pointed out that I neglected to take into account information available in a 40-second video on the site. Now, I don’t agree that a 40-second video is ENOUGH to change my conclusion, but at least the guy offered a bit of evidence! He added something to the conversation.
Would you like to add something to the conversation, too?]
Hi James. This seems to be Automatic Software Repair technique being applied. Doesn’t look like bullshit at all 🙂
[James’ Reply: There is no information about that tool on the website. THAT is BULLSHIT. They talk and say nothing. THAT is BULLSHIT. You don’t know what techniques are being applied, unless you have some other source of information.]
So if a UI element locator changes, this tool would let you know that there is something breaking up your automation, and if you accept it, would apply the fix automatically.
[James’ Reply: Your description bears little resemblance to the claims on their video.]
We were flirting with this idea with the TestingAR meetup team in Argentina for a while, but never had the time/drive to implement it.
You can read more about this here: https://www.monperrus.net/martin/automatic-software-repair
Btw, I agree that this tool is not replacing any “Smart QA person”, it would just replace a a UI automation developer doing boring maintenance fixes. I didn’t tested the tool though.
[James’ Reply: How do you know what MABL does? Where did you discover it?]
Cheers from Argentina
There are people who still outsource development to the lowest cost nameless contract house. Why? Marketing and people with only a passing understanding of what they are trying to purchase. If it works to sell junk development it will work to sell junk test tooling. It’s not terribly surprising right?
“teeming with sentences that definitely contain words” is a lovely turn of phrase btw
My absolute favourite quote from the CEO of Codeship “Nobody on the team has to worry about testing”. Now maybe this is an amazing tool but I can’t imagine that’s the best quote to be attaching to your brand.
I am talking about the video right next to the “Maintain Tests” label.
Well, in that 40 seconds video there is a claim of fixing UI locators when they break and how they harvest the DOM (theoretically, of course), find fix candidates by finding differences in locators and then proposing a fix. That’s feasible to apply with Automatic Software Repair techniques.
[James’ Reply: I see that now. It’s not nearly enough information, and it is packaged as a fluffy video with a tool that pretends to be a human. But it is more than zero, that’s true.]
I’m not aware of how it works internally though! It’s just a guess. I mean, it’s not impossible. Would you like to try implementing it with us?
[James’ Reply: I’m a tester. I study and practice testing. That’s not testing. But I’m happy to talk about any tool that can help testers do testing.]
Also, I think this sort of automated software/tests repair is better suited for maintance than for creation of actual tests, yeah
Twenty years or so later and we are still dealing with Snake Oil (and myths and misconceptions in general). As an “Automation Guy” I have my doubts and concerns, and I might be pleasantly surprised if it lives up to some of its promise. But it is a tool only and cannot replace the computer between your ears. And as the saying goes “A Fool with a tool is still a fool.” I’ll reserve full judgement till later on when it is being used in a real world situation. Hopeful for a new tool to utilize, but skeptical until I see it for myself.
[James’ Reply: Lots of test tool makers offer ridiculous, vague, sweeping promises. What shocks me is the complete failure to explain what the tool does to anyone who is a technical professional. It reminds me of this commercial from SNL. ]
Actually I prefer this one: https://youtu.be/2HKTx5WFcs0
Unfortunately the video cuts out before the best part.
“Mmm… That’s Good Bass!”
Hi James,
I’m sitting here thinking how did they test this tool to see if it actually performs the job that it says it does? As you said there are no QA professionals in the video they introduce the tool with. As such by who’s professional opinion does the tool even accomplish all that it says it does?
I notice as well that they say MABL is able to find all the broken links and changes and reports these. I’m curious to know if someone is also manually checking that this is the case because any self-respecting QA team would know that no software product is defect free. So is anybody verifying these results or are they simply trusting these results at face value?
Hey James,
I found a pretty funny bug with that self proclaimed testing site. I’ll email you a screengrab of it.
It’s something any “really smart QA engineer” would pick up on very quickly.
-Eoin
[James’ Reply: Thanks for that!]
I’ve never even heard of MABL until now, however from my look on the site their blog articles do appear a bit more descriptive of what they claim the tool does.
https://www.mabl.com/blog/how-to-create-test-scripts-without-code-using-mabl
https://www.mabl.com/blog/maintenance-free-automated-functional-testing-with-auto-healing-tests
https://www.mabl.com/blog/machine-intelligence-for-clearer-regression-testing-insights
It appears to be just one of those “scriptless automation” tools with some fairly nifty html object recognition and analysis of regression tests thrown in.
I got a chuckle from the stuff on the whiteboard visible between the person wearing a sweater over a shirt. (Add that to the list of their sins.)
The upper left has a short (and shallow) overview of “Test Driven Development”: “Red: create a [test]; Green: make te[st pass]; Refactor: improve [the code]” – the parts in brackets are not legible but easily reconstructed.
The bottom left has a list of first names, possibly team members.
But what was fun was the upper right, where I could recognize things like “sellIn” and “Backstage Passes” – it’s a whiteboard analysis of the Gilded Rose Kata. (https://github.com/emilybache/GildedRose-Refactoring-Kata).
Other than that, I kind of disagree with you here, James – I can all too well figure out what kind of tool this is. Based on the docs (https://help.mabl.com/docs/defining-a-plan-for-mabl-to-test-a-journey) this looks to me like a bog-standard capture-and-replay tool that periodically come back into fashion. I’d think of it as basically Selenium, updated for the times with a Slack integration. Yet another data point supporting the common contention that the industry is fashion-driven, with the bulk of what is labeled “innovation” being in fact a perpetual recycling of the same old ideas under new covers.
[James’ Reply: How did you find that documentation? I don’t see a link to it on the main page, and the guy I chatted with didn’t tell me about it, either. Why wouldn’t have have steered me there when I asked for details?]
Since we’re on the subject of documentation. Check out what you should do if one of your tests fail 😀 https://help.mabl.com/docs/troubleshooting-test-failures
Where is the automation??!
Well, I was curious about the product, so I clicked the “Try For Free” button (green, top right of the home page, also left of the button that starts the video).
[James’ Reply: I was also curious about the product, but I didn’t want to try it. I wanted to read what it does. Because I know how to read, and I value reading as a method of learning. Apparently the people who created this website do not, or they wouldn’t have designed it to have the depth of a children’s book.
Meanwhile, I engaged their sales guy in an online chat, during which I said I wanted details, and he did not see fit to point me to any of the documentation that you mention.]
From there, the “Get Started” button took me to a page where it looked liked I’d have to provide personal information, and I didn’t want that, so I went back and chose the other button, “Read the docs”.
Clever of them, hiding it in plain sight. (I’m saying that semi-seriously: I can see how the video might have been a distraction and led you not to take the other informational signposts at face value.)
[James’ Reply: “Try it for free” is not an informational signpost, it’s a commercial signpost. I assumed “for free” meant that I had to download something or sign-up for something.]
(Two side notes about your blog software, James: each of the two comments I’ve left here contained at least one error that I felt embarrassed about afterwards, but I see no obvious way to edit my comments once I’ve submitted them. And a related problem: it’s not that I submit my comments in haste. I know this, because I click the “I’m not a robot” check box when I think I’m ready, then typically I reread my comment one last time to check for something to add or correct. By the time I’m done, the check box says it’s “timed out” and asks me to verify I’m human once again. If I *wasn’t* human maybe I could get it right the first time and submit fast enough for its short span of attention.)
[James’ Reply: Usually people write a comment asking me to make an edit.]
I felt that if I am going to form an opinion of my own I should look in to the product, and I have to say that I was curious when they said “Mabl can provide the quality hay teams need to free up the developers to…” so I downloaded the free tool, filled out the information using a google account, and clicked the verification link in the email. After I finished clicking the link in the email, I noticed that I could login using google, so I clicked that button…Remember how I said I used a google account as my account information, so I would think that I could log in using Google, only to find that the page just reloaded, forcing me to login manually. To add insult to injury after I logged in the tool then grabbed my google account information.
The testing joy just got better after that, I used a testing account to use with my cool test I was going to create, only to have the super cool tool get hung u on the log in page. to make a long story short I logged two bugs with the company.
+100500 )))
Agree with you 🙂 Now many people trying to somehow “apply” AI to automated testing^ but no one understand that “magic” is only in their minds.
So, in next 10 years, we probably will have some tools which will be able to test some software (not all possible software, but some web sites, for example) using AI and will find some common issues on such web site. But today is too early, at least because people don’t know how to use AI in the right way.
Did Mabl miss a typo on the MABL site 😉
On this page I found the below statement
https://www.mabl.com/features
The monitor panel is your main dashboard which shows reportd of test execution history, test statuses, and recent insights from your tests.
Unfortunately it’s not that suprising or shocking that companies are taking a “swiffer” approach to Test Automation. I am guessing that is in response to the how demanding financially putting Test Automation in place can be and so we’ve now entered the “save a buck” approach which will only end in disaster since this simply cannot be done with testing. So I guess some companies who sign up for this app might be in for some major disappointment and a rude awakening! That’s if of course it lives up to its presentation. Let’s just hope the marketing dept is more horrible at doing their jobs than the app itself.
I can’t help but wonder how much more interesting a interesting blog post this would have been, if someone at Satisfice had taken the time to contact the MABL people, get a demo of the product, and *then* review it in as much harsh detail as they cared to muster.
Instead, what we get, is yet-another boring engineer’s rant about the marketing gloss technique. Something that’s been around, and annoying engineers of all stripes, for almost 100 years now. Too bad.
[James’ Reply: Wow, you took the time and energy to complain that I didn’t write a blog post about something I have absolutely no interest in, while ignoring what I did write about. I suppose you could go to anyone’s blog and do that: go to a blog about politics and complain that they wrote another post about Trump instead of contacting the people at MABL…
This post is not about how tool companies exaggerate. It’s about how it’s reached a point of 100% bullshit with no information. They are selling tools by offering less information about the product than you get in a Viagra commercial (which at least lists side effects).
I’m sorry you don’t care about this, but some people do.
I suppose what I should do is ignore you, Greg. But I will point out that if you feel that it’s important to be completely constructive and carefully consider all the information that you possibly can before you write anything in public, then wouldn’t someone at gmgauthier.com have done that in this case, instead of writing another boring comment that ignores the point of the post he commented upon?]
Honestly, this has me scared. I’ve been hearing the hype about AI taking over testing, and have (in polite company) laughed it off. But this MABL product — if AI is able to do the bullshitting without human intervention, then testers are truly in danger.
Testing and quality assurance of software has been at best a cargo cult or kabuki dance as far as business and development is concerned for most organizations. If you can feel like you have performed the rituals or dazzled for most organizations most of time.
I haven’t been around quite as long as you, James, but surely you realize this too?
If you believe testing doesn’t really matter — and I’d love to hear you talk about one organization where it does — then if a dashboard with a feminized AI makes you feel as warm and fuzzy or satisfies your due diligence, then even if it costs more than the traditional performance in manpower, the savings in square footage and HR headaches dealing with actual people are more than worth it.
As Confucius say, “sincerity is second only to faithfulness in performing duty”, and he was really just putting an optimistic spin on Cicero’s “uphill both ways in the snow” argument. You sound more like the Roman curmudgeon.
But in order to make sure the AI is doing it’s job (while we can let go of all the testers) we should keep the test managers in place to make sure smart AI like MABL continue to do their job efficiently and accurately, and besides, we need someone to handle vendor relations and contract renewal.
[James’ Reply: I don’t understand your point. It sounds to me like you are trying too hard to sound clever and not hard enough to understand what I was trying to communicate.]
They promise “infinite scalability” – how was that verified?
I clicked on “Pricing”, but there was no pricing. The FAQ on the site actually answered some basic questions, and was better than that video.
There are a lot of companies in the test automation business that do “hyperbolic summarizing”, my favorite is how the term “scriptless” gets tossed around. Many of the tools that brag their product is scriptless simply abstract their steps into cells, and overall just make it harder than if there was a high level script to work from. Another classic thrown around term is “artificial intelligence”, which most of the time ends up being a smart recorder.
It never ceases to amaze me, James, how you continually offer your experience and insight for free, and a lot of people reject it.
This is a cautionary tale that James is discussing in this post. To this day I fight against “record and playback” mentality in businesses, mostly with inexperienced testers but also their ignorant superiors who are always looking for any elixir that will reduce costs, increase productivity. What’s lost in these marketing messages is the reality that there are elements of truth to what is being sold, but at the end of the day, nothing beats experience and intellect.
I often remember a blog post you made back in …. oh something like 2002, where you described this amazing automation harness that your team had created. It would run 24×7, it would re-start where it left off if there was a power failure. It generated reports and mounds of data …. and you were left looking at these heaps out output that told you nothing useful about anything. That lesson of yours helped to shape how I looked at this type of thing forever more.
Thank you for that, James. Don’t give up, you’re not alone. #BuccaneerScholar
[James’ Reply: Thanks, man.]
Roll up, roll up we have a fantastic offer for you. This pure snake oil wonder testing cure will ale all ailments and cure all cures. No need to think, it just works as it says on the box.*
Wait be quick to take up this offer and one of the first fifty testers and you will have an exclusive additional free license and support.
*results may vary from clinical trials.
We had a marketing call with the guys from MABL and they kept throwing shit into Selenium – only by hearing this and I won’t go with their solution. If your marketing relies on demonizing a tool who set out a standard in the market … and then you ask them how they select an element … well, I don’t care they propose a fix if the selector is broken. There are better straightforward solutions to this problem: ask the development to add a test attribute to the elements i.e. and boom, never again will your test be broken because of selectors. So, instead of applying a good practice to solve your problem, you pay thousands on a crappy tool in order to spend even more f***g time to understand their selectors and approve changes through their tool. You solve your problem by generating a new one, which takes same amount of time to fix it as in the first place. And then I asked what’s their wait mechanism – and they even don’t have async support as of today (15.08.2019). And then I ask what’s their queueing mechanism between test steps, and they tell me “the best would be to add assertions for elements on the screen to know you are ready to go to the next step” hell yeah I can do that in any open source tool out there in the market …
This say-nothing marketing-speak bullshit is everywhere – it’s where the industry is and has been for a long time: in the toilet.
Nice to hear someone call it like it is and stand up to (and swat) the lemmings.
Keep it up. You’re an inspiration.