No more Voodoo SysAdmin allowed for Tech Support organizations

In my past life as a systems administrator and tech support person, one of the terms I remember hearing about was "Voodoo sysadmin" -- which I basically think of as doing something to a system when you're having a problem in a vain and superstitious hope that it'll fix the problem, without really understanding what's going on.  (An old acquaintance from that life, Mark Verber, has an article about Voodoo Sysadmin (among other things) if you want to learn more.)

This frequently takes the form of "it's acting up" -- "reboot it, and try again".  Sometimes, this solves the problem, at least temporarily.  A lot of other times, it doesn't solve the problem, but it doesn't do any real harm.  Put those two together, and you get a classic recipe for superstition: accidental reinforcement (more on wikipedia, or in Karen Pryor's book, Don't Shoot the Dog).  That is to say, because it works some of the time, you're bound to try it almost all the time, just in case it works.  Often, this is mostly harmless, if perhaps wasteful of time.  Sometimes, it creates real harm.  For example, rebooting a UNIX system that's having problems because some critical file got corrupted may leave you with no way to log back in and fix it, without getting out installation media.  Or worse, it may remove the temporary copy of the still-working version of the file that was lying around until the reboot process cleaned it up.

I fear, though, that I'm getting overly geeky for a point that's more universal, so let's instead go to a real-life example from my very recent past.

I was having a problem with my iPhone syncing its photos to my Mac.  Image Capture was saying "No camera or scanner connected."  iPhoto simply didn't have a "Devices" section under which it would show up (until and unless I connected another digital camera, which would show up fine).  Aperture 3 wasn't showing it either, though it had had partial success earlier (and Image capture used to work).  After a long conversation with Apple (and paying the roughly $75 for AppleCare protection on my Iphone, since I was beyond the original 90 days of phone support -- I'd started the call as a Mac call, which was still under warranty), a Senior iPhone Advisor named Kurt told me that there was a known issue wherein sometimes certain images that were downloaded from online, or from e-mail, or maybe even saved from apps on the phone other than the Camera app, would somehow get corrupted (or at least some meta-data about them would), and this corruption would cause a situation wherein the various photo-related apps on MacOS X would simply fail to see the phone as a device with images on it.  This was totally what I was experiencing.  They're working on a long-term fix for this, and in the mean time the workaround given to me was: Email myself any images not created by the Camera application (screenshots, downloaded images, etc.), and then delete those images from the phone.  Note that this does not involve doing a hard reset on the phone, or rebooting my mac, or, and here's the kicker, resetting all settings on the phone, which is exactly what a previous associate had asked me to do (and all the other things were asked, too).

Resetting all settings did not help.  It did, however, cause me some general annoyance at having to restore things like my ring tone of choice, and enabling caps lock, and that sort of thing.  And to re-enter wifi passwords (I'm glad I knew off-hand the main one of those that I care about; I'll have to go and find the others again, as they become relevant).  But that's not what really got me.  It also deleted all of my alarms from the Clock application.  Uhm, I rely on those to remind me to take some medications each day.  And others to make it to regular appointments.  I noticed this a couple hours after I was supposed to have taken a dose of medication that I take daily.  Now, I take it daily, and being a couple hours late isn't a big deal for the medication in question.  But what if it had been something I had to take on a very regular schedule?  Or what if I hadn't noticed that the alarm hadn't gone off, and I just missed it today completely?  Or missed the appointment that I have later in the day today?  Who's to say what would have happened.  I'll say, though, that it could have been bad news, if not for me than for someone else who didn't realize they'd lost their alarms.

So, step one was to complain to Apple about this, and ask them to make sure that their associates are all trained to make sure they let a user know that their alarms will go away.  Had I known that, I could have made a list of them first, before resetting all settings.  I've called Apple, and spoken to another Senior Advisor, and he seemed to take it all fairly seriously, so I have hopes that good things will happen there. Better yet would be to have the UI for resetting settings actually tell you this -- or maybe even be able to turn on and off which settings get reset.  This, too, has been suggested to Apple.

What would really make me happy, though, is for them not to have asked me to do something that was totally unnecessary! Resetting my settings didn't help, and I'm sure that while the guy I was speaking to suspected that it might have, the reality of the situation was that he didn't understand why my phone wasn't being recognized, and thus didn't know if it would fix it or not.  Maybe he knew he didn't know, maybe he thought he knew and was wrong.  Either way, the fact is the same: he didn't know, and so he went with a superstition-based or "voodoo sysadmin" approach to fixing the problem. He even claims he resets all settings on his own iPhone every week, as a matter of course.  If that's not superstition, I don't know what is.  If I did that, it would drive me nuts...  I have a lot of settings changed.  And a lot of alarms -- the important ones of which I think are all back, though I know I had others in there that I'll have to re-create from scratch (ones that were off because they weren't set to repeat, but which I would occasionally turn on for certain things -- now, I'll have to actually re-create them when those situations arise again, instead of just turning them on.  I can live with that.

But here's the thing:

If there was a concerted effort within the Tech Support industry to try to eliminate all superstitious practices from their support calls, this kind of thing simply wouldn't happen.  And I believe it shouldn't have had to happen.  Because, as this blog is all about, I believe there is a better way.

"Eliminate all superstitious practices?  Shyeah, right."  I know, I know, it'll never happen.  See Verber's article (linked above): superstition is part of Human Nature.  True.  I have no argument there.  Still, it's the effort to eliminate them that would bring about the change that I want.  As I see it, there would be several main components to such an effort:

1. Educate tech support personnel on what superstition is, how to recognize it, and how to avoid being trapped into it.
2. Teach these same folks alternative ways of doing things so that they can actually find actions to suggest that will be known to be helpful.  Now, this will be impossible in some cases, because they just won't have a way of figuring out what's wrong, which brings me to item #3:
3. Have software (and hardware) developers provide better instrumentation in their products, and analysis tools which can either be used by support staff, or given to end users to run on behalf of the support staff, with results being given back to them in the latter case.  Also involved in this is more and better error reporting, and/or more use of any extant error reporting by support staff.  Many of these tools could be built in to applications.  Others would be separate tools.  Either way, more troubleshooting would be helpful.

If Image Capture had given me some sort of message saying "iPhone detected, but the image database looks to be corrupt", or if there'd been a menu option for "Detailed device detection", or a help item for "Why doesn't my device show up?" with instructions on how to run some debug information, or something, I would have kept my alarms, had a lower degree of frustration, and possibly even saved both Apple and myself some money, by not having had to buy AppleCare (yes, I get it, that *makes* them money), and not having them have to take the call (this is where they get it back -- I was on the phone with them for a while, bouncing between different people, and calling back the next day to let them know of the problem I had with the service I'd gotten -- that may or may not add up to the cost of the plan, but I bet it came close, at least).

And so...

In summary:

Software developers:

Instrument your code, and provide tools for analyzing problems as they're going on.  (Example of a fairly decent (if under-technical for the hard cases) version of this: The network connection assistant in Apple's network preferences on MacOS X.  It goes through each stage of trying to get online, and tries different things, giving you things to try along any stage that's showing difficulty.

Hardware developers:

Uhh, I dunno.  I'm not a hardware guy.  But something like the above.  And make sure you work with the software folks to build the tools to make use of the instrumentation you're providing.

Tech support managers:

Train your people on the perils of superstitious support behaviors, and reinforce them when they go through the admittedly often-more-difficult process of actually trying to figure out what's wrong.  (Oh yeah, and this probably means training them how to do that, too, and rewarding them for doing it, and costing you lots of money in more advanced employees.  I bet you it's worth it in the long run, though -- actually fixing the problem without negative side effects makes for happier customers, less likely to call back again about the same problem, and more likely to exhibit loyalty to your brand.  But that's just speculation on my part.)

Thank you for reading.  I hope this is somehow helpful to someone -- even if only for having listed the workaround to the iPhone connectivity problem, so the next person hitting it can fix it themselves, and save themselves a bit of heartache.

Wishing for a better world,

  David

P.S.  Oh yeah, and the medical industry could probably do a lot of this, as well.  But that's another rant, for another day.  See Karen Pryor's book for a brief discussion of this point, when she's introducing the idea of superstitious behaviors.

Date and time stamps in all blog/news articles

This post was written on 2010-02-05 (That's February 5th, 2010 for those not familiar with ISO 8601 -- and please, make yourself familiar with it, and use it -- but that's another post), at around 14:31h Seattle local time.

I expect that date to appear above the title of this post, and the final time of me publishing it to appear down below.  Automatically, and in a fairly easy-to-find way.

It would be great if all web articles did this.  Now, I'll grant you: I've been guilty in the past of occasionally omitting this detail in some web articles of mine, and there are some that need to be corrected to include it.  By and large, though, those articles have not been of a news variety, and tended to include only information that I generally considered to be relatively timeless (though really, anything can change, so a datestamp is really still quite desirable). Recently I was reading a news-style article, though, that referred to something having happened "last week".  Well, I came across this article from a source other than the front page of a news site.  As such, I had no implicit idea if I was looking at an article from today, last week, or several years ago.  I was still on the top screen-page of information, in the second paragraph of the article, so I glanced up above, looking for a date, to try to get my bearings.  Nothing.  I then proceeded to scroll to the bottom, to look for a date.  I couldn't find one.  I did eventually find one in the meta tags on the page.  2009-12-11.  OK, so about 2 months ago.

Surely it would be better to have this information written plainly at the top of the article (ideal), and/or at the bottom of the article (less ideal if "or", more ideal if "and").  That way, (a) readers who have no idea how to look for a meta tag would still be able to find the information, and (b) readers (like me) who do know how to do that will be able, instead, to just glance up while still mid-sentence and grab a context for what they're reading, and continue without having to be interrupted to write a blog post about how frustrating it is that there wasn't a date.  :-)

So, please, news purveyors, bloggers, and really any content creators, please: provide datestamps, at least, and preferably also timestamps (sometimes that matters -- especially on the day the item was published) on your articles.

I will endeavor to make sure that all of my future content is done this way, too, and perhaps retroactively add it to some of my old content.

Thanks.

(2010-02-05, 14:43h local)

P.S. (2010-02-05 15:02 local) -- I just e-mailed the webmaster of the site in question.  Hopefully they'll fix their site.  Yay, reporting problems, so that people might actually have a chance to realize their error and fix them.  Though again, that's probably another post.

MUAs and MTAs should talk more

I will not go into great detail, here, though perhaps requests in comments would get me to say more.  For now, though, the short version:

I have come to hold firmly the opinion that Mail Transfer Agents (MTAs) and Mail User Agents (MUAs) should talk more.

For example, it should be possible for me, from my MUA (currently mutt, though I'm thinking of switching to something emacs-based) to flag a particular message as spam in a way that permanently blocks all messages to the recipient that that mail was destined for.  Now, obviously (I hope) this only is relevant in cases where one is using a wildcard accept type of rule, i.e. where any username at the domainname in question will get delivered to one particular actual user's mailbox (with perhaps some exceptions going to different users, or, as implied here, being rejected as spam).

Now, perhaps folks just aren't doing this very much any more, and I should get over the idea that all MTAs everywhere should have this capability.

For what it's worth, I wish at least Gmail for domains would do it...  and have something procmail-like, too...  or rather, something more elaborate than the current filtering rules, which I find limiting.  Though, perhaps I'll give it a go, we'll see.

(I'm currently in the market for a new mail infrastructure.)

An End to all Spam

The world would be better if all Spam (which I'll define below) would go away.  All of it.  Period.  Forever.

Spam, as I mean it:

- Any e-mail communication which has been sent unsolicited as part of a bulk mailing.

It could be commercial or not, and still be spam.  It could be bulk or unsolicited and not be spam.  But if it's bulk and unsolicited, it's spam.  That's my personal definition.

And it should all go away.  I don't know how many hours I've spent battling spam -- either as an e-mail administrator in a corporate IT department, or for my own personal mail system.  Regardless of the actual number, suffice it to say it was Far Too Many Hours.

Spammers, please stop.

And if you get spam (and of course you do), please...  PLEASE, I implore you: Do not EVER buy something from a spammer.  Don't do it!  Buying something from them encourages the practice.  And the fewer people, ironically, that buy from them, the more strongly it's encouraged (to a point; read Don't Shoot The Dog for more on that).  So don't do it.  Don't.

And spammers, stop spamming.  And companies thinking about spamming, don't.

Don't Shoot The Dog - Teaching, Training, and general psychology

The world would be better, I assert, if everybody on this planet... or at least every teacher, parent, manager, or anyone else having any influence over others, would read and internalize the book Don't Shoot The Dog, The New Art of Teaching and Training, by Karen Pryor.

Seriously, everybody should read it. And internalize it, which probably means reading it numerous times, and playing "The Training Game" a lot, and... who knows what all else.

We frequently get our interpersonal interactions in ways that are pretty ineffective at actually getting things we want, and in ways that the other person (or creature, or whatever) will be happy with, too. And there are a few core concepts, discussed in this book (among other places, I just happen to like this particular presentation of the material), the understanding of which could make a world of difference in how we interact with each other.

It would make us happier. We'd never hate our jobs again. It would end divorce. It would bring world peace.

Perhaps these claims seem outlandish. I honestly believe, though, that if a bunch of people read this book, we'd actually go a long way towards each of those.

So go to your local library or book store, or even order a copy or twelve online if you must, but someway get your hands on a copy. And read it. And find someone to play the training game with.

First person to send me a private message mentioning this post and giving me a shipping address -- anywhere in the world -- gets a copy sent to them. (If you know me in person, claim it in person, and save me the shipping. :-)

The world will be better if I share my ideas...

I've come, just moments ago, to a conclusion: The world will be better if I share my ideas (with the world). This means I'm going to blog here about some ideas that I've long been considering to be more or less "trade secrets" or something. Inventions of mine, in concept, that I think the world should have. And frankly, I'm never going to make all of them happen. And I'm also never (until I die, that is), or at least I hope I'll never, stop having more of them. So sharing some now won't preclude me from making a billion dollars off them later. Well, it might keep me from making a billion dollars off the one I share, because someone else might go do them before I do. But if I'm not going to do all of them anyway, I might as well let that happen, and hope that my life can be improved for having access to the thing I wanted enough in the first place to dream it up. That said, if I happen to get any readers that are in a position to fund a startup to make some of these ideas fly, I'd be happy if you'd drop me a line before starting something going. Chances are good that I've thought about this (whatever idea it happens to be) in more depth than has made it to the blog, and perhaps for some stock in a startup, I'd be happy to be on an advisory board or something. Or maybe, for some of the ideas, I'd even be up for launching the company myself, with the help of your financial resources (and, then, by hiring the requisite number of Really Smart People to make it all happen -- hopefully, the Done, and Gets Things Smart sort of people. Sorry I keep picking on you, Steve). But really, I think the world will benefit just as much, and probably more, from someone else taking these ideas and running with them, than by helping me to run with them. I'm overburdened or something. (Which goes into a whole different story, which will likely find its way into this blog in some form or other over time. Today is not that time.) So yeah. Any idea that I don't have an expectation that I'm particularly likely to try to put into practice on my own is now fair game for me to blog about. Here's hoping the world will benefit from it. This all begins with the next post.

What Could Be Better - an introduction

Inspired by a multitude of factors, most recently Steve Yegge telling me I should (the reading of which can be indirectly attributed to an e-mail from my friend Goose), and next most recently by the movie Julie & Julia (imdb), and third most recently by... oh, I forget. But it's not important.

The point is, I'm starting a blog. On how the world could be better. This blog will be a collection of ideas, rants, and other such musings on how I think the world could be better.

Some will be about a piece of software. Others about photographic equipment. Others on Human behavior. Others on... Well, time will tell. I'll just be writing on whatever ideas I happen to have (and be warned: I have a lot of them), related to making the world a better place.

It's kind of my life's mission, actually, to show the world that there are better ways to do things. This blog will be a way for me to work towards that mission.

Thank you for visiting. Your thoughts are welcome, on this or any post. I reserve the right to disagree, agree, or just take them as food for thought. Regardless, they will surely all be reinforcing (more on that later, like as not).

Oh yes, and one final thing, about the URL of this blog. Like "everything is better with butter" (a sentiment I tend strongly to agree with in many many cases -- though I won't go so far as to say all), I believe there's a lot that could be better in this world -- and sometimes by just a simple addition of something basic. Like butter. So, "better like butter". Besides, all the other sensible variations on "what could be better" were taken.

Thanks,

- David