Mint: The Flavor Of The Month
Next week is a big news week. Apple is releasing their new lust object, NFL season begins, and there are tons of Labor Day concerts and festivals to attend. But the biggest development for me personally will be the release of The Wolf’s shiny new creation:
Mint.
For the last several months, Mr. Inman has been frittering away his nights and weekends creating what I believe to be the most useful stats program in existence. It’s not Urchin, it’s not Analog, and it’s not designed to record every single hit to your website since the beginning of time.
But that is its strength.
Mint, as the name so cryptically implies, is designed to give you a “fresh look at your site”. A slice in time, if you will. Where has my traffic from the last 48 hours come from? What days of the week does activity on my site peak? What terms have people possibly been searching for that could lead them to my embarrassment-of-a-website?
Mint grew out of a desire to maintain a rolling window, or dashboard, of activity with which one could easily analyze what’s going on in the here and in the now. Not so much in the two years ago. It is this philosophy which allows Mint to maintain such a small database (generally around 20 megs, but fully customizable) and yet provide such great functionality.
I’ll let Shaun spill the full details upon release next week, but I wanted to briefly talk about my favorite Mint feature; one I feel somewhat responsible for since I badgered The Wolf so incessantly about it all through development — The 24-hour “Drive Through” Referrer Window.
Repeat referrers — the single most useful web stat
As a previous user of both Shortstat and Refer 2.0, I get great value from perusing my list of referring URLs. For the uninitiated, this is a list of all the URLs on the web which people are clicking on to get to your site. Did MSNBC just link to a blog post of yours? Bam, it’s in the referrer list. Is Metafilter sending over morons to your latest iPod contest only to have them suggest “Google.com” as a potential “greatest site you’ve never seen”? Again, it’s in the referrer list.
Most stat programs, including Shortstat, will give you a linear list of referrers as they come in. This is only marginally useful because so many of them are repeats. Other stat programs, like Refer 2.0, will group referrers by how many clicks they receive within an X hour window. This improves the functionality tremendously.
However, the biggest problem with referrer logs lately has been the onset of referrer spam. Essentially what happens is that spambots hit your site and pass along a fake referrer like “http://unlimited-free-viagra.com” in hopes that you, as a referrer log junkie, will click on the link and purchase (?) some free Viagra. The referrer spam problem has gotten huge lately. So much so that on any given day, my Refer 2.0 readout shows 80% referrer spam.
Mint to the rescue
What The Wolf has done with Mint, however, solves the referrer spam problem outright. Since Mint validates every hit with javascript, no referrer spam ever gets through. In other words, if you aren’t using an actual browser — one with javascript enabled — your referral hit will never even make it into the system.
Purity in referrer logs. Is there anything more beautiful?
So anyway, stay tuned for the impending release of Mint. You should hear more about it and its many other features in the next few days.
Dang it’s good to know people isn’t it. I can’t wait to check out Mint.
After reading your excerpt in NNW I immediate checked my email to see if The Wolf had sent out an announcement to prompt this post. Sadly I found no such email.
Regardless, thank you for the minuscule glimpse into Mint.
I feel like Awstats already does all this as is. It even gives you more features. I think I might just be missing something.
j: “More features” than what? I only mentioned one feature of Mint in this blog entry.
I hope Mint will be better designed than Awstats :)
Just from screenshots alone, it seems that Mint will be worlds ahead of Awstats design-wise.
Looks like it’s going to be pretty awesome.
Useful referals. That’s crazy talk!
Incidentally, does it track internal referers via the same minty magic?
j: AWstats tries to do so much, I’ve always felt like the important things get lost amidst a bunch of big numbers that seemed to only matter to folks making media kits (not to devalue marketing people or anything of the sort).
Ooooh, I can’t wait. Shortstat went all wonky on me and I’ve been waiting on this for awhile. Well. Even before that, since we talked about it in Vegas last year.
Oh and I love how you just had to show your metafilter and msnbc hit counts. Whore. ;0)
(Editor’s Note: Haha. I was waiting for someone to call me out on that!)
I think all the important numbers get lost amongst the unimportant numbers, whereas Shortstat, and soon Mint hopefully, show just the stuff you want to see. My 2p, at least.
Also, “The Wolf”?!
I help run a Technology related Community featured by MS and I have been waiting for mint to come to my rescueto make it simplier to show who is coming to the site and form where.
Excellent. As a stats junkie I’m far too enthused about Mint. My opinion, there can never be too many stats programs. It’s just nice that finally we’ll have a visually pleasing one.
I can’t wai to see it! The first stats harvester that actually looks great.
Also very nice to see that Shaun implemented my javascript solution in order to prevent those pesky spammers. Actually I improved it a bit more lately which is why referrer spam is a thing of the past now.
I only hope it’s not going to cost too much money…
I, like everyone else, can’t wait to see this. Finally a time-frame though. More info in a few days. I wonder how much it’s going to cost.
Just don’t tell me it uses MySQL ;p
Pure referrer logs – my god how I’ve longed for that. The amount of poker referrers I get makes me wonder why I never started playing cards professionally…
Why would you want it to use MySQL Faruk? I in fact pray it won’t. These days just about everything uses MySQL, often for reasons unknown. Shortstat’s doing fine without it and I hope Mint will follow the same trail ;)
Doh… I should read a bit better first…. *hangs head in shame*
ignore my comment ;)
Any word on the price? or is it Freeware?
Mike,
Appreciate the update you, Jeff, and Keegan have offered us today! One week, you say, until the Wolf released Mint?!?! Fantastic!!
Oh, and loved the reference to the kid’s website… Man, you had me reading his, yours, David Shea’s (and a host of others) whole posts again concerning validation. I know I’m behind the times on this subject, but have quickly been trying to learn from as many sources as possible as I make a transition from Graphic Design to doing more web development. What was supposed to be a quick note of thanks turned into an hour and a half of study before going to bed.
This is looking really awesome. I figured the interface is going to be kickass. It would also be sweet if you could custom skin Mint’s interface with CSS to incorporate it better into an admin/backend section of a site.
I didn’t know how Mr Inman was going to innovate in the features dept, but he seems to be doing just that. The repeat referrers feature is a great commonsense idea.
He just needs to get this bundled with some hosting cPanels over Awstats and Webalizer. If he can make it easy for people to switch to Mint, I have no doubt this thing will take off.
OK.. OK.. I have heard enough minty non-sense. I don’t mean to put down any buzz for what sounds like it should be a fantastic traffic analysys program. But, what does it REALLY have to offer that can’t be found anywhere else? I mean I really want to know?
I will be sure to try it out.. There is obvious excitement around Shaun Inman’s latest creation. But where are the real facts. Where is a REAL feature list and where for the love of stinky cheese are the server specifications?
Is this a Linux only creation? Multi-Platform? Is it 100% JavaScript and stores data through the use of magic fairy dust? Will there be a cost (large or small?)??..
Mike, don’t make me reference stinky cheese again.. Someone spill some more details to me. Really, I have a good fix on what Shaun is capable of producing and am geniunely excited about learning more even if it’s not something that I have little use for right now.
Mike, BTW, assuming those referrer counts are accurate that you have there. I am in awe at your ability to drive visitors your way. I have worked for a couple companies that had huge amounts of traffic because of how large and well known the company is.. But driving large amounts or traffic like that to a blog which is just your random thoughts and an occasional Ipod give-away is quite impressive. Is it just your contest that draw in numbers like that?
There must be a devious marketing scheme behind this product. Create a nice logo, blog about it from time to time. Leave it hanging. Have some scenesters do the beta-testing, tell them they can blog about it from September 1st on and fuel the hype some more. All for free. Very good. ;)
I have to see how Mint set itself apart from its competitors. Ok, it looks far better than anything I have seen so far but I wasn’t that blown away by Shortstat. I hope to find both a simple but also complete application which is affordable to boot.
@Collin: There’s no time frame visible in terms of Mikes repeat referrers, could be a weeks worth of traffic, could be a month – it’s not actually telling us anything.
Also, what’s with the secrecy surrounding the cost of this confectionary? I’ve noticed a few people mention it on this and other sites, and no one seems to have any answers. Mike, do you have any info?
(The following is more a question of principle than anything else. Basically, what’s being described as filtering I’m seeing more as obfuscation. For the duration of this comment, let’s just assume you happen to run a site where my examples are a consideration.)
In other words, if you aren’t using an actual browser — one with javascript enabled — your referral hit will never even make it into the system.
Doesn’t make it into the system, or into the reports? It seems like this knowingly introduces some amount of inaccuracy1, and it might be good to have the option of seeing everything if you want/need. It’s been ages since I’ve used it, but as far as I can guess from your phrasing, Mint effectively denies the existence of Lynx, and also of people who turn off Javascript. If your server’s being hammered into submission, it sounds like you wouldn’t be able to determine where it’s coming from.
I’m assuming the official statement on this will be something along the lines of, “Then maybe Mint isn’t the option for you,” but I’d hope there’s some clear disclosure about it.
1. Maybe untruth is more correct, since I suppose it’s still accurate in terms of what is being shown.
Su> spam referrers completely screw up your webserver stats. On many, MANY sites spam referrers would take up 80% or more of all visitor stats. They completely defeat the purpose of any stats program. I realize using javascript penalizes some hits by lynx, search engine bots or people with javascript disabled but it’s a lot better to miss 1% of your stats than have 80% added by spammers in my opinion.
Still, you can use a second stats analyzer in order to identify the spammers, lynx users, non-javascript users and search engine bots, preferrably one that actually parses the http acccess log. AWStat for example. Both Mint and this second kind of stats analyzer have their own (different) purpose.
Marco: I understand that perfectly, but it’s not my point at all. Again: principles. I’m concerned about presentation. Is the user told, and how?
To restate my initial concern differently, and probably better, filtering output(the report) is a very different thing from filtering input(the hits).
While I’m fine with the marketing saying “Mint gets rid of referer spam” I think it’s important to explain someplace that what really happens is that Mint ignores anything that doesn’t support Javascript, including the case of someone turning it off.
[Note: Assuming the above is correct. And yes, I know there’s speculation here, but if it’s correct, then there is time to consider before going public. For all I know, the logic actually does a quick check for Lynx immediately upon failure of the JS check, and then lets it through. This is an easy test for the beta peoples, if they want to chime in.]
I would want to know if people were hitting my site with Lynx. If Mint does in fact completely deny that Lynx exists(by not saving the data, leaving it impossible to access, or however else), I personally consider that unacceptable, and I had damn well better be told up front.
Yes, a secondary stats program is a good idea. But the user has to know it’s a good idea, and possibly even be told why. Also, people who use multiple stats programs don’t use them in isolation. There will be discrepencies due to the quirks of particular packages, but if AWStats says I’m getting some noticeable amount of [whatever] traffic, and Mint says absolutely none at all, something is clearly not right. More importantly, the assumption is most likely to be “Mint is broken, and I wonder how much it’s screwing up the other stats.” Disclosure cancels that out.
Just so nobody thinks I’m trying to kill this thing before it’s born, I’m only objecting to this one somewhat arcane detail. It looks lovely, and if there’s a free trial, I’ll give it a poke. I’m supremely disinterested in stats for my own use, but it looks like something that would be great to recommend to clients.
Thanks for the extra explanation Su. And yes you’re absolutely right. It should be made very clear in Mint’s documentation that it (probably) doesn’t log lynx, non-javascript visitors and visiting search robots.
When I come to think of it, another issue are mobile users. I own a dating & chat site which I built myself, strictly targeted at mobile phone users (i-mode to be more precisely). Mint would (if our assumptions are right) log absolutely NOTHING on that site because hardly any mobile phone available right now would run the javascript. ;)
Jakob: Nope, it doesn’t track internal referrers, but that would be dead easy to bake in. I’ll ask Shaun. Perhaps there is already a “Pepper” for this.
Faruk: Yes, it uses mySQL as its database. Since the stats are rolling, your database never gets too big, so in the case of this product, there are really no advantages or disadvantages to using one database over the other. Heck, even if it used Oracle, it’d perform about the same. Quick.
Everyone: With regards to price, I can’t really divulge that right now, but it’s definitely in the “extremely affordable” category.
Collin: Yes, those are real referrer stats from a 24 hour window. Most of the time, I get several thousand page views a day, but occasionally, a place like The New York Times, Kottke, Metafilter, or MSNBC will pick something up and I get a little bump.
Pascal: Mint will not cure cancer for you. If you like it, you like it. If you don’t, you don’t. It’s aimed at developers, designers, and average citizens who want a nice easy window into their site’s activity. It’s not aimed at webmasters who need to know every single Apache transaction in order to sleep at night.
Su: Yes, currently Mint uses javascript to validate all hits, and yes, this has the bad and good effects that both yourself and Marco have mentioned. The bad is that, for most people, your traffic will probably be about 1% low. The good is that, for most people, your traffic will not be about 30% high. This isn’t the product for you if you run a web site where any serious percentage of your traffic comes from mobile phones or text-only browsers like Lynx, but again, what did I say above? This product is for most poeple… not all people. If you honestly get any serious percentage of hits from Lynx or mobile devices, then there are other stat packages which may suit you better. No product is perfect for everyone… this product is great for most though.
Fair enough. Will shut up about it now. I think my fixation on details tends to get overinterpreted, and I’m pushing my luck as is.
Who cares what database it uses if it’s a hosted service? It’s Shaun’s problem, then *grin* I thought half the point of hosted services was that you don’t have to worry about what goes on inside the magic box.
hmm… I wonder if my Pathstats plugin for shortstat made it into the mix.
Mike,
Well, noting first that I’ve no idea what kind of queries Mint runs, but MySQL isn’t great with constant inserts & updates and also selecting at the same time. In fact, on very high profile sites, it’s quite horrible at that, even.
That’s my only concern with it using MySQL. Since stats tend to be a constant stream of inserts, this is something I’m concerned about, and since Mint apparently keeps only a limited set of data, I’m assuming there’s also constant pruning and/or updating of data.
Still, it’s unlikely that a blog will get the kind of traffic where this becomes an issue. Something like Boing Boing might, but they’d probably just use replication if they were to use Mint..
I’m also not thrilled with the fact that it needs MySQL. I’ll still buy it and use it if it turns out to be nice enough but in my humble opinion a stats program doesn’t need MySQL. In fact there are many things that use MySQL as a backend because it’s the easiest way out from a programmer’s perspective. Being a programmer myself I know that often it’s much easier to bang stuff in tables than to store it in a textfile based database because you have the power of SQL at your fingertips. However it doesn’t mean things can’t be done without it. A fine example of what you can do without any SQL database is Pivot which is the blogging software that powers my site. It can do just about the same as Movable Type and WordPress but there isn’t any SQL database involved. BBClone, which is the secondary stats program I currently use besides ShortStat also isn’t using MySQL.
I guess to me this is the first ‘bad’ thing to hear about Mint. Just my 5 cents of course!
Frank: You might be able to go MySQL still on a high traffic site by switching to InnoDB for the Mint tables. If you know ahead of time you are going to get a hammered database table, it’s always good to just switch it over from the start.
Mike: “Pepper” and “baking”… this is still software, right? ;)
Having used ShortStat for a while (mainly lured into using this because of the buzz it was surrounded by) and later on abandoning it because of the size of its database and its narrow feature set, I am very impressed on how much propaganda one can achieve by doing web marketing right.
Most clever marketing hype ever.
Me bows. Shaun Inman is one of the web’s savviest communicators with talents widely exeeding just coding and design.
Jakob,
It’s Faruk, and yeah, that may alleviate the stress on the db a fair bit.
The growing list of beta testers is at least highly impressive. Most well known names are there. I guess I’m just still waiting for Jeffrey Zeldman and a couple of others but most of them are here. So yes, Robert’s definitely right about the marketing hype. It’s getting more and more amazing. I hope I’ll ever be able to gather such an awesome PR team together if I ever release some paid software ;)
What is Apple’s new lust object?
Thanks, Mike, it’s just a stat program I was looking for! I am using SpeSta, which is actually good, but doesn’t provide me with some useful information I hope to find in the Mint. I hope it will meet my expectations and something inside me tells me that I will find one of the most usable and accessible stat systems ever written. Thanks again! :)
To Mike and others who are concerned or NOT concerned about the use of MySQL:
Mint demo page:
Of course, I can’t say this with any bit of certainty at all, as I simply haven’t used Mint myself (yet?) at all, but to me, this just screams classic MySQL incompetence. If the system gets crippled from hundreds of hits a minute, it’s either MySQL being poor or the server is severely overloaded from other things.
Faruk: A “hit” to the server in this case is not just a hit to the server. What’s overloaded things is that over 100 users a minute are making multiple full queries to the Mint database. This would theoretically never happen in a real circumstance because it’s just you and perhaps several other people who may be checking a particular site’s stats at the same time. I don’t know enough about mySQL or any other databases for that matter to say if things could be made better with other software, but this really is not indicative of Mint is used.
Mike,
Entirely true that this doesn’t quite reflect generic Mint usage. However, my experiences with databases so far has indicated that PostgreSQL would’ve handled such a situation much better. I won’t go into the lengthy details on that unless requested though (it’s likely to become quite a story once I start explaining it properly).
Just to clarify something: I’m a PHP and Database programmer by profession, and have been for my entire career. XHTML, CSS and design are just “extra stuff” for me, even though I’ve come to a point where, nowadays, I do more of the latter three than the former two.
I can’t wait to see Mint. The load on the demo, while extremely unlikely in real-world useage, is surely quite predictable in this scenario. Perhaps Mr Inman could put up a few static pages instead of the live demo?
I was dissapointed to learn that MINT will not work for me on my site.
I must look upon the masses with a sense of envy in my heart and the comfort that no one visits my site anyway. hehehe.
Best of luck to “The Wolf”
J.
This looks awesome. All the famous people are using it :)
I was sort of surprised by the fact that curl is a necessity to run Mint, only to discover it’s used for the license check. I’ve rewritten the check to do exactly the same (it still checks the license key) but without needing curl.
I sent the code to Shaun. Hopefully he’ll like it because it enables more people to run Mint ;)
Marco: and warez it. Good luck getting him to like it ;)
Faruk> I’d never warez it. Why would I want to hurt a fellow developer?
I didn’t DISABLE the license check, I changed it to not need curl anymore that’s all. And I’m not disclosing my changes to anyone but Shaun himself.
If people want to warez this it needs just a couple of keypresses. Shaun is perfectly aware of this, being a PHP developer. But let’s just hope people are going to play fair.
Marco: ah, okay, I misunderstood what you’d done, then :)
It worked out rather nice. Mint is now cURL-less ;)
Our Stats program over at http://htmlfixit.com used Javascript two years ago. In fact we then added a non-javascript read, but in a way that has yet to result in comment spam, to pick up the bots trolling thorugh. Ours is either free or $10 depending on which features you want. It might not be the mint, but it still tastes good to a whole lot of people.
This? (iTunes 5 as well!)
Is this really a new thing or only a redesigned old idea? Please don’t get me wrong, I always like to see development and usability is a very thing. But as this doesn’t seem to be freeware it looks like a usefull but not really necessary thing to me. Please correct me if I am wrong. Thanks.
I dont know what does it mean
Well, if you want something similar to Mint that *is* freeware, try this out: http://www.w3counter.com
Mint is Coming
Apparently, Mint is going to be released soon–next week, perhaps. For those that don’t know, Mint is the successor of…
[…] Davidson, “Mint: The Flavor of The Month” – It’s not Urchin, it’s not Analog, and it’s not designed to record every single hit […]
Mint is awesome, but it’s not free. I’ve been working on an open source version Mint-like web statistics program, called Grape:
http://www.quate.net/grape
[…] Mike Davidson — Mint: The Flavor of the Month […]
[…] Inman’s Mint, by Kevin CornellMint: The Flavor of the Month, by Mike DavidsonMint: Better than Girl Scout Cookies, by Jeff CroftMint: Fresh ‘N Yummy, by […]