A second new blog — Christian Sepulveda’s Blog

Chris Sepulveda has also been working on setting up a new blog lately. I’ve spent time with Chris at the Austin Workshop on Test Automation last February and more recently at Agile Fusion. He’s the coach on an XP team that’s in a different location from where he is most of the time, which gives him a different perspective on XP. I’ve been impressed with his thinking whenever he and I have talked, and I’m looking forward to reading his thoughts on his blog. I actually am planning on responding to an entry he’s got posted on integrating testers into XP, but that entry hasn’t quite finished simmering in my head yet (nor have any of the post-Agile Fusion things I’ve thought about, for those waiting for them).

Chris’ blog is at http://www.christiansepulveda.com/blog

New blog — Thoughts on Pragmatic Software Quality

My wife now has a blog of her own — Thoughts on Pragmatic Software Quality (at http://blackbox.cs.fit.edu/blog/heather).

She and I have similar interests, but where I’ll in general turn a discussion towards the more technical side of things, she’ll lean more towards the business and management side. I’m looking forward to reading what she has to say, and I believe that those who find my blog interesting may also share an interest in hers as well.

I’ve been helping her get her style sheet and rss feeds set up correctly.

New search feature

Because I can’t resist tweaking as I learn about new things, I’ve changed the search functionality in my template for this blog. Thanks to Micah Alpern (found via Stuart Henshall and his Unbound Spiral blog), you can now search the entire web, just my blog, or all the blogs that I read. Currently, the blogs I read list is generated manually through an OPML export from SharpReader. I imagine that in general, this will be enough to keep it relatively current.

I also experimented with having the blogroll links also come from this file (and I may yet be able to do that, but it’s not going to be quite so easy as I had hoped, I fear). Anyhow, my experiments led me to delete my original blogroll (multiple times). That meant the recreated one had a different id, so if anyone actually used the RSS feed for the blogroll, you’ll need to update the link to the new one. Sorry about that — moving forward, I don’t plan to delete the blogroll again, so hopefully this will be the last time I need to change that RSS URL (or maybe no one uses that feed and I can change it with impunity). Those of you reading this through the web site will notice a few other changes too — the list is now alphabetical (I didn’t want to recompute the priorities that I had in there before), and there’re more entries (again). Since the list is starting to get long, I’m pondering switching the list to show links in random order and then telling it to stop after 50 links. Any thoughts on this? Anyone care?

And so it (temporarily) ends

This is the last post I’ll be making from Front Royal. Those who may have gotten used to multiple posts a day will find themselves sadly disappointed in the upcoming months. There’ll be no posts for the next few days (I’ll be without net access as I drive back home then spend the weekend with my wife and in-laws). I’m planning to think about this week as I drive, and I’ll use a voice recorder to keep my thoughts. There’ll be more posts on the topics from this week coming as I digest and think about all the subjects we’ve covered. I expect other people may be writing about this, and I’ll provide pointers to that writing as I hear about it. I also want to pull together a post about my experiences specifically in blogging the event live. I learned some things, there’s some things I probably will do differently next time, but that’s another topic for next week sometime.

It’s been a good week. I’d like to thank Lenore and James Bach for being such wonderful hosts and all the people who have participated (either directly, through comments posted here, and even just reading these posts and thinking about the subjects) in this workshop experience.

Guest post from James Bach

One of the things that I (Andy) don’t feel like I conveyed well in this blog is some of the conflict that came out in our discussions. I think my failure in this regard stemmed from two pieces — the conflict aversion I described a few posts back, and the fact that this was a group of passionate people. As conflicts became apparent, passions rose, and it became harder and harder to keep up with the discussion at the level I needed to remain a participant and succinctly summarize each person’s viewpoint and the tenets of their arguments.

James read my Wrap-up post and felt that I inadequately portrayed the conflict we had this morning specifically. I reworded part of the post to remove the obvious place where it could be interpreted that we were all in agreement on everything. I also invited James to write up a guest blog post about his feelings on the matter. Here’s what he sent:

It’s very important to understand that we in this Agile Fusion
conference did not come to agreement on *any* specifically phrased idea
about what agile testing is or isn’t. We attempted to discuss it, but it
quickly became apparent that there are important philosophical and
terminological differences among us. Brian finally made the point, and I
strongly agree, that it’s the experience of this conference, this week,
that matters. We have experienced things. We have become exposed to new
ideas and skills. There was indeed a lot of friendly cooperation as we
worked on code together– and that is enough for now.

I think that the exploratory testing we did during this conference might
be misunderstood and misapplied, in the future, by people who attended
this conference. But I’m not really worried about that. At least they
experienced some of what ET is all about, instead of merely seeing a
synopsis on the web. I’m much more worried that someone will read our
pronouncements about ET and get the wrong idea. What may not come
through in those pronouncements listed in Andy’s blog is that we are
speaking from a position of lightness, openness, and a spirit of
learning. We are not trying to construct the next Capability Maturity
Model or ISO standard.

I learned enough about test-first programming and refactoring, during
this conference, to know that I have significantly more to learn before
I consider myself reasonably competent. I considered myself a reasonably
competent programmer prior to this week, but now my standards are
higher. I hope that the programmers who came here this week now have a
similar impression about skilled testing, especially skilled exploratory
testing. There’s a lot to it, and we just scratched the surface of the
subject.

Naked/Ironman CRC

Michael F. gave us a demo of what he calls “Naked CRC” or “Ironman CRC”. This is a technique that he uses to explain systems visually, using motion and spatial relationships. It’s a bit hard to explain completely textually, so if you ever get the chance have him demonstrate it for you.

Basically, the technique calls for using blank cards put out on a flat surface, grouping them and overlapping to illustrate concepts. Cards or groups of cards can be moved around as necessary to illustrate concepts. This description feels even more vague now that I’ve written it out than it does in my head. Basically have someone who has seen the demonstration demo it if you’re curious.

Wrap-up

Cem prepared this list to help us discuss the outcome of this workshop. We spent this morning discussing and refining it (with John moderating, which I mention so that he can get his Toastmaster’s credit ). All of these things are in the context of testing on an agile project, where people typically shift among many roles. The “tester” might have a job title as programmer, tester, or something else, but when she wears her testing hat, this is what she’s up to. These extend and complement the points posted in an earlier entry.

• Exploratory testing is “simultaneous learning about the product, designing tests, and executing tests”
• Agile test groups exist to provide test-related services that add value to the project.
  Agile test groups provide valuable services to programmers, customers, and other stakeholders who may vary from project to project.

• A core purpose of testing is to expose and investigate product and project risks. Achieving this typically requires application of a diversity of testing techniques.
  Skilled testers serve as the headlights of the project.
  Caution: Agile testing is more then free-form exploratory testing

• Collaboration among testers, programmers, and other stakeholders is more highly valued on agile projects than details of process, practices, or tools
• The agile tester’s objectives will vary across projects, and as the work evolves within projects.
  Examples:
  • Help programmers understand the program as they create it
  • Create change-detector suites to facilitate refactoring
  • Help project management understand the state of the project
  • Find bugs
  • Comply with regulations

  Each of these may dominate the thinking of the test team at different times on the same project

• There are noteworthy similarities in approach between XP and exploratory testing. Both:
  • Emphasize and cherish skill, integrity and individual’s best judgement
  • Break tasks into timeboxed chunks that are explored and handled by a task owner (with help). “Stories” and “Testing charters” are very similar concepts in practice
  • Limit expenditure on document creation to what is demonstrably needed
  • Confront the learning curve through stucturing workflow to allow for change as we learn more
  • Emphasize ways to keep the brain continually engaged when the person is doing technical work on the project
• A distinguishing characteristic of the agile test team is their degree of support for the interests and desires of the programmers they collaborate with.
  Testers perform services to many different types of stakeholders but on the agile project, they increase their focus on services for programmers.

• Skilled agile testers protect programmers from information overload. They apply judgement to the questions:
  • What issues should I raise with the programmer?
  • What issues should I investigate soon, before raising a subset with the programmer?
  • What issues should I begin to monitor but investigate later?
• Two of the vital services provided by agile testers are:
  • Development of tests that help guide/coach the task (specification by example)
  • Development of test suites that expose problems caused by change, especially by refactoring (change detection suites)
• Within XP, the primary goal of the “acceptance test” is to facilitate understanding (communication):
  • Acceptance tests draw attention to the essence of a benefit, and guide the programmer in the design of the benefit-enabling code
  • Other tests might be better designed for regression(change detection) or as scenarios intended to explore a broad range of risks associated with this benefit in the context of the larger program
• Automation, ease and rapidity of use, and maintainability are essential attributes of tests designed for change detection.
  These attributes may be much less important for tests designed and used for other purposes.

• As agile testers develop trust of and insight into the practices (including tests) of programming teams, they abandon or slash their investment in tests that are redundant with programmers’ work or for other reasons unlikely to provide much value.
  This might free the agile tester from most boundary tests, for example, leaving them more time to explore hostile attacks, write experienced-user scenarios, translate stories into acceptance tests, or build better change detection suites.

• Skilled testing often involves extensive preparation and research. It is normal for this work to be an ongoing activity rather than something completed early in the project.
  Like everyone else on the team, testers know less at the soing development of risk liststart of the project than they will know at any later time.
  Examples:
  • Config testing including lab setup, identification of relevant combinations, ongoing research on compatibility risk
  • Development and use of complex applications of the software under development
  • Ongoing development of risk lists
  • Bursts of detailed analytical work, to build models or other bases for selecting among complex tests
  • Ongoing development of tools
• The prime purpose of a bug tracking process is to help the team get the right bugs fixed. An informal process that achieves this process is often good enough.
  Introducing formality in order to acheive other purposes, carries several costs. Cost-benefit analysis and evaluation of the plausibility of actually acheiving those other benefits are appropriate.

• Much of the technology of agile testing has been created as free software. Whenever possible, we should find ways to contribute to the community repository.

  
  Bret added:

• Exploratory testing is a process that conforms to the principles of Agile Development
• Exploratory testing naturally complements XP

  
  James added:

• Skilled exploratory testing can be a powerful complement to unit testing and customer acceptance testing on agile projects

  
  Brian:

• XP is a bet and the bet is that generalism trumps specificism (People switch roles as opposed to specialists)

  
  Cem:

• When you advertise for a programming job, you are already advertising for a specialist – maybe not a specialist among programmers, but a specialist none the less

We want to stress that we’re not ready to make industry-wide grand pronouncements. We don’t have the experience yet to do that. These are hypotheses that some of us believe, but there was quite a bit of controversy over some of the points as well. Further discussion and experience is required.

Quotes from nearby

“Can you put a lizard in your head? Yes. Is putting a lizard in your head better than cheese?”
“We’re all zombies on a train”
“So, we’re not actually using the brains we have now”
“So, in addition to stopping on a brain, you can also stop on other people”
:”you might choose to leave a completely useless brain in someone else’s head”

Just thought I should cover the full range of conversations going on here…

Dinner conversations

A few tidbits from dinner conversations:

• First, a couple of nights ago, Bret was talking about psychological barriers that hold us back. The example he gave was that maybe he wasn’t as successful as a developer because he didn’t want to redesign people’s code. This got me to pondering. I’m generaly prone to avoid controversy. In fact, when I started out as a tester, doing automated testing, i used to say that it let me create things but no one criticized my code, plus there were a lot fewer people doing test automation well. The latter half was definitely true then, but now I’m wondering if it’s more this confrontation avoidance that keeps me from really feeling like a tester — if I’m truly a tester, I have to criticize other people’s code, something they’ve put effort into. Will have to think more about this.
• Then, tonight, a bunch of us went to The Main Street Mill restaurant in Front Royal. One of the topics that came up was the future of testing as a career path in its current form. Bret maintained (as I have heard Cem maintain in the past) that testing as it’s currently defined will become a dying profession. Instead, testers will have to get better programmer skills and/or business analysis skills. They’ll find themselves pairing with developers more and so either path will become more prevalent. All the testing skills will still be needed, but, as Jeremy pointed out, the lines between tester and developer need to blur. The testers who have good developer skills will work on creating the code, bringing their testing skills to bear for exploratory testing and developing good unit and acceptance tests. The testers who get more business analysis skills will take on more roles towards being customer surrogates.
• The idea of having tasks specific to testing came up as well. The idea is that you specifically have tasks like “test the xyz thing” in which the pair consciously changes their mindset and takes more of a testing approach than a development approach. This task would not necessarily be done by testers, particularly as the lines between the roles blur.
• Chris talked about testers becoming more of a wild card on the project, taking on different roles as needed. He commented on how testers are generally the only ones who get the more global knowledge of the application rather than focusing on a specific area.
• Brian had a comment on the role of testers. He said that the role of testing is to “articulate the perceived status of the software”. That is, if we were to release the software right now, what would customer’s perceptions of it be.
• Finally, we talked about what kinds of things testers need to know to fit in on agile projects more. We hit two topics before sidetracking in to less reportable ordinary conversation — patterns (and, specifically, it seemed to be the Gang of Four patterns) and that it’s ok to pair.

  Anyhow, that’s some of the points we talked about at dinner.

More observations

* Brian observed that it was interesting pairing because it seemed that at any given moment he wasn’t as confident in the code and his understanding of the code as it has historically been on his own (but as long as the sum total of confidence was ok, he was ok with that). He attributed it to test first design — in the early 80s, he’d think lots about the problem before writing code and the code would just flow out, but now when he does test-first programming, he doesn’t have as full an understanding of the code because the tests understand the code for him in some ways. Mike said the understanding is still there, it’s just written down in the tests. You don’t need to keep it all in your head, and when you need the understanding you don’t have in your head, you can go back to the tests and “re-inflate” the parts you need.

* He also observed that we got caught up so that the code was beginning to drive user level decisions until he pulled the pair back. He finds that same cycle in his own personal programming — coding distracts him from thinking about what the user would want and he needs to consciously pull himself back. He asked if there was a certain art or skill to diving into the code, the code tells you what it can do, then you go back to the customer level, and back and forth, having it all come out nicely. Mike said taking breaks sometimes has been helpful for him. Bill also suggested keeping the customer in the room helps with this question of focus.

* Mike H. observed that installing stuff is a big pain in the butt. He was surprised at the number of things he had to install and configure in his work today on the install procedure. Mike mentioned an interesting thing to consider — how many projects start the install stuff in parallel with the application development. He thinks there might be some benefit to doing that.

So, now we’re breaking