Metacritic has fallen on hard times

Ever since I first became aware of Metacritic and Rotten Tomatoes, which I think was about the same time, I have been steadfastly a Metacritic supporter. That's even though Rotten Tomatoes is by far the better known and more culturally referenced method of compiling critic and user reviews to reach a single score that represents the value of a cultural item.

It wasn't a methodology thing. In the past I have looked into how Metacritic does things and how RT does things, and if anything, Metacritic utilizes the more mysterious, inscrutable means of producing its scores. I believe they have even come under fire for being less strictly data driven or less accountable to explaining their procedures.

Really, it came down to what it comes down to in a lot of these situations for me: Presentation.

I always liked how Metacritic looked -- its fonts, its layout, its whole gestalt. Meanwhile, I thought Rotten Tomatoes looked like it had been made by a child who had mastered web design -- a very competent child, but nonetheless, one who liked childish things. Like I have never gotten into the tomato splats. They just look like something you'd see on Nickelodeon's official website.

I'm not saying I feel any more favorably toward Rotten Tomatoes than I once did. I almost never visit the site.

But neither do I almost ever visit Metacritic, which has become a shadow of its former self.

The first problem I noticed a couple years ago was that the site was absolutely crippled by ads. On any page you tried to reach, there would be at least one video and usually a couple other Java-powered ads scrambling around to get your attention. Sometimes you'd have to close ads just to get to your content, but the ads themselves would contain false icons that made you think you were closing the ad when really you were clicking to learn more information about the product. Or there would be multiple ads for the same item with the content sandwiched in the middle, in a reading area that was truncated from its previous real estate.

I can be frustrated when a thing like this happens, but I also understand it. The reality is, it takes revenue to run a website. If you don't have other streams of income, you have to rely on advertising more and more. And the advertisers knew what they were doing, making their ads increasingly pernicious and increasingly debilitating to the usability of the site. (According to me, they did not know what they were doing, because any ad you perceive as pernicious should not be effectively selling the product. But it must have worked with some people because you still see these sorts of ads.)

I don't notice Metacritic being destroyed by ads these days as much as I once did. Maybe the advertisers wised up, or maybe Metacritic wised up. Now, it's an issue of functionality that has nothing to do with the effect of the ads. 

When I liked No Time to Die but was curious to see whether I liked it more or less than other critics, I went to Metacritic to get a sampling of their opinions. And this is what I found when I tried to search for it:

One of the biggest movies of 2021 cannot be found on a site dedicated to reviews of culture, through a normal search of its title.

Now, I know No Time to Die has a page on Metacritic because I googled it. That took me to the page and allowed me to see that the movie has a 68 score, which is almost exactly aligned with my own 7/10 (3.5 star) rating I gave it on ReelGood. In fact, I hesitated a little between 6 and 7, never seriously considering 6, but knowing I had enough qualms that a 7/10 was not a slam dunk. That uncertainty on my part computes to a 68 Metascore almost exactly.

Now why couldn't I have just found this by searching the site?

I suspect it has something to do with the lack of distinctiveness of the four words in the title. Those four words would appear in countless other titles, and it's possible that they just don't bring up the film when taken in combination with each other. A major malfunctioning of the site's core search architecture, to be sure, but one that at least has some sort of explanation.

But let's conduct the search from the very page itself, something that does not change the way the search performs, but just looks funny in context.

Sorry, that's a bit small, but maybe you can zoom in, or just wait for me to explain what we're seeing in the next paragraph. 

In this particular instance it does give some results, as it is searching all of Metacritic, rather than the focused movie search I did previously. The titles it does produce are not only significantly more obscure than the movie I'm searching, but they also feature common words that would appear in multiple titles. If my previous explanation for the failed search holds water, then why is the 2014 iOS game "No one dies" able to be found? Those words are certainly no more common in frequency in the database than "No time to die."

My goal today is not to roast and lambast Metacritic. It's actually far more solemn than that. It's to mourn the solid product I used to love and to visit on a weekly basis. Nowadays, I might go months between trying to use this as a resource to gauge critical consensus on a movie, and the latest James Bond movie is a reminder why.

The sad thing is that I'm not going to RT either. I'm just not looking up these critical consensuses, something I used to dearly enjoy and consider a key aspect of my engagement with the movies. 

Our relationship with any form of culture evolves over time, but when that change is forced by an external failure like this one, rather than something essential about our own changing perspectives, it's disappointing indeed.

Slap me right in my face why don't you

It’s not every day that I am put in my place, but every once in a while it’s funny to have a thing like this happen:

In case you can't read what it says, I will retype it, to drive the stake further into my own heart:

"DEREK ARMSTRONG

Derek Armstrong's reviews do not count toward the Tomatometer. This is not a Tomatometer-approved critic, and this critic's reviews are not published on a Tomatometer-approved publication."

It was something I found when googling myself, and no, I don’t google myself very often. (When I do I get a lot of hits from a retired NHL player, and they can be annoyed to wade through.)

It relates to two reviews that got posted to Rotten Tomatoes (not by me) when I wrote for All Movie Guide between 2000 and 2011. Two random reviews, it would appear: Lymelife (2008) and The Living Wake (2010). 

The wording of it sounds almost like a warning, like “Don’t take any wooden nickels! Don’t take any opinions from a critic not recognized by the Tomatometer!”

I knew there was a reason I prefer Metacritic.

RT's unexpected boosting of Oscar nominees

I hope you haven't been coming to my blog in the hopes of getting hyped up for the Oscars, because I've done a piss poor job at that. I write about what I'm thinking about, and I just haven't been thinking about the Oscars much. It's consistent with a shift in my thoughts about the Oscars that's a good decade old now. I'm always interested to see what gets nominated, but then I don't expend a lot of intellectual energy on it again until I'm watching the show ... at which point it's very passive intellectual energy.

But I figured I ought to write at least one preparatory post, given that this will post on the same date as the Oscars (though the Oscars are not really until tomorrow, if you're reading this within the first few hours after I post it -- it's an Australia thing). And it has to do with this article on Rotten Tomatoes. (You don't have to follow that link, as I will summarize the article below -- I just wanted to give credit where credit is due.)

A lot of people have a lot of feelings about Rotten Tomatoes, but one of the loudest seems to be that the site will lead to the ruination of the film industry. By reducing a creative endeavor down to a number, especially one that reflects only a yea or nay mentality and not the shades of gray that better describe most challenging pieces of art, the film in question is cheapened, and maybe ultimately not seen by people. Which leads to fewer of that type of movie getting to be made in the future, and in the end, their ultimate extinction. So more or less was the argument of Martin Scorsese when he wrote a much-passed-around op-ed piece on the topic last year.

I don't particularly frequent Rotten Tomatoes because I prefer its competitor, Metacritic. I don't know that the numerical rating is any more scientific of a method of assigning value to the movie -- especially since Metacritic has put forward some weird language related to accountability for their own inscrutable formulas -- but I can tell you that I vastly prefer the interface and the layout. Or used to, anyway -- some recent changes have made it harder to get out from under the ads, and navigate in general. Even in that compromised state, though, I still prefer it to the perennially childish looking fonts and logos on RT.

Yet an article on The Verge gave me a fuzzier feeling about Rotten Tomatoes and the influence it's having. In at least one way, it's not quite as toxic as Martin Scorsese and others would lead you to believe.

You know how the last two best picture winners, Spotlight and Moonlight, were actually considered by some critics to be the actual best movie of the year? A critic I highly respect, Michael Phillips of the Chicago Tribune (I know him from his guest appearances on Filmspotting), chose each of those as his #1 movie of their respective years. If Lady Bird wins this year, he'll be aligned with the Oscars three years running.

Which I imagine may cause him some sleepless nights, but shouldn't. See, it's no coincidence that the Oscars are "getting better." You can certainly credit the recent changes to the Academy membership, and we shouldn't overlook that. But that article on The Verge also posits that it has to do with Rotten Tomatoes, and in the next paragraph I'll tell you why.

Now that knowing the RT score for a movie is kind of an inescapable part of the cinematic zeitgeist for people who follow movies, that can certainly be considered a bad thing. But it can also be considered a good thing, in that Academy members will feel more sheepish about voting for nominees that have lower scores. Some Academy members are mavericks (or old people) and will still vote for whatever they want, but others, who have their eyes on the bigger picture of the Academy's stature as a respected institution, will consciously vote for films that they know critics love as a way of boosting the Academy's credibility. And that is easier to do than ever when you know that films like Lady Bird and Get Out are bursting at the seams with near universal acclaim, each boasting a 99 on the influential site.

I won't rehash the entire article, as I'm already conscious of the fact that this is sort of what I'm doing rather than adding my own unique take on it. But suffice it to say that they went back to examine the scores of previous best picture winners from the earlier part of this century -- before RT existed, but you can still retrofit their scoring system to existing reviews of those movies -- and found that best picture nominees and winners both had a lower average Rotten Tomatoes score then than they do now.

Of course, you could argue that there's some echo chamber stuff going on here that poisons the statistics. At least some of the people who reviewed these newer movies were already conscious of their near universal praise at the time they reviewed them, and did not want to be the only critic submitting a rotten review. (Though three brave souls did for Get Out, four for Lady Bird.) But I think when analyzing this stuff, you have to take it as a constant that critics are paid (or not paid, as the case usually is) to give their honest opinions of movies, and we can and should trust that they do.

And whether it totally works as a correlation between Rotten Tomatoes and the better field of Oscar nominees, I think we can say for sure that this is happening. There are only a few "stodgy, old-fashioned" choices for best picture this year, and they are actually my favorite two: Darkest Hour and The Post. Maybe that just means I'm starting to resemble an aging Oscar voter, or maybe it means that only the best Oscar bait is rising to the level of nomination-worthy. Darkest Hour and The Post are at 86% and 88% on Rotten Tomatoes, respectively. If they have the support of critics, who usually try to see through transparent Oscar baiting, then they are, by critical consensus, more than just standard Oscar bait.

And there is indeed a perception shift starting to happen, I think. Not only were a diverse slate of nominees nominated again this year, but you don't hear people grumbling as much about how the Oscars don't matter and that they always nominate the wrong films. There are some best picture nominees I don't love this year, but I don't think you can argue that the films nominated were something other than the ones that showed up most often in top ten lists. Even something like Phantom Thread, the prototypical example of a critical darling that got nowhere near the best picture race in the past, shattered its own eccentric art film ceiling this year.

Okay, hope you enjoy the ceremony, and as you're watching it, appreciate the fact that you are witnessing a rejiggered Oscars about which we may no longer need to roll our eyes. It's an exciting era for an awards fan. Relish it before something comes along to break it again.

And as for me, to try to start my own Oscar engines, I may finally go to see Lady Bird tonight. It'll be my last chance before the big show. They say The Shape of Water is most likely to win -- a shock if I've ever heard one -- but I still have a sneaking suspicion it might be Lady Bird, which will make a very funny pairing with 2014 winner Birdman. And if I do see it, it means I can be sure I'll have seen the best picture winner before the ceremony, as it remains the only one I haven't gotten to.

Metacritic's imperfect math

It's finally happened.

A movie on Metacritic has reached triple digits.

I'm sure Boyhood is not the only movie on Metactric with a perfect score of 100, and it's not likely that it will always have this score. But for at least long enough for me to take this screen shot, it had reached those ludicrous heights of critical adoration, the likes of which I have never seen when surfing to the site.

Except, it's totally bogus.

I talked to a friend, who has already seen the movie (jealous) and loves it, about the improbable 100 scored by Richard Linklater's universally acclaimed masterpiece. And he was the first one that got me doubting its authenticity.

In an email earlier this week to me he said "How could it be a 99 a week ago? Doesn't that mean that there was 1 mixed review at least? Did they discard the mixed review? Did the reviewer clarify the review, reclassifying it as a positive?"

Good questions.

So I looked at the numbers offered up on Boyhood's main landing page, and indeed, they were all 100s except for one glaring exception: a 75 (from Slant Magazine's Ed Gonzalez) gumming up the whole works.

But with only a single dissenter out of 40, that 75 might still not be enough to reduce the film's average below 100. So I did some quick math. Assuming all the other reviews were a 100, and Ed Gonzalez' review was a 75, that would be 3975 divided by 40, which equals a 99.375 Metascore. Rounding down, as you are compelled to do in this situation, you get a 99. Ed Gonzalez himself is able to keep this movie from having a perfect score.

But tonight I delved in a little deeper. On Metactric, you can go past the landing page and see all the reviews and all the scores that are currently being tabulated for this movie. And it was here that I found that the movie did not have 39 reviews of 100, but rather, "only" 32. That is still jaw-droppingly astonishing, but it also meant that even a Metascore of 99 was likely too high for Boyhood.

It turns out Boyhood also received scores of 95, 90, 90, 90, 88, 83 and 80, in addition to good old Ed Gonzalez' decidedly contrarian 75. Some more quick math: 3891 divided by 40 = 97.275. So not only does Boyhood not deserve a 100 or a 99, it doesn't deserve a 98 either. It deserves a 97.

Why the fuzzy math, Metracritic?

Scanning Metacritic's FAQ, I think I may have my answer in this dismissively pithy response:

Can you tell me how each of the different critics are weighted in your formula?

Absolutely not.

Simply put, not all critics are equal on Metacritic. And some -- many, in fact -- don't get any love at all. Take Armond White. He's a pretty prominent critic, albeit one whose contrarian views almost destroy his ability to be taken seriously. True to form, he has a negative take on Boyhood, an excerpt of which was recently posted in a group I frequent on Facebook:

"“Hipster Patriarchy” might be a better title for Richard Linklater’s Boyhood. Depicting a white American male from childhood to adolescence, it celebrates the emblematic figure of American social power."

Needless to say, White was not one of the 40 critics who passed muster for Metacritic.

I visit Metacritic almost daily and prefer it to its most similar rival, Rotten Tomatoes. But some of the opaqueness I'm seeing in the FAQ, particularly the flippant way it is expressed, makes me see Metacritic a bit more as something elitist and capricious than something open and honest. Why can't we know you're formula? What's so great about your formula, anyway?

There's some kind of snobbery or something going on here, a form of preferential treatment for some critics over others that requires the whole thing to be shrouded in mystery. Metacritic's role in taste-making, therefore, seems sneaky and underhanded, something for them to be ashamed of. Only if you are a certain kind of critic espousing a certain kind of view can your opinion carry its full value in our system. Seems more than a little bit fishy.

Consider the tone of this part of the FAQ:

Why don't you have 97 reviews for every movie like those other websites do?

Several other websites that provide links to movie reviews have weighed the quantity vs. quality issue and come out in favor of quantity. These sites typically include links to as many reviews as there are available on the net. And lately, with every Joe Schmo posting a movie review both before and after movie releases, there are quite a few reviews for each movie (we're talking 100's of reviews for the more popular titles). True, some of these Joe Schmos--or at least the Harry Knowles--do have quality sites with useful reviews and information. But the quality of many is inconsistent at best. In addition, there is such a thing as too much information, and statistically, once we include a certain number of reviews in our calculations, adding additional reviews will not change the overall METASCORE much in one direction or another.

It's funny that I can essentially agree with what they're saying here and still be appalled that they're saying it. The Joe Schmos they refer to -- and if they are so elitist, you'd think they'd opt for the spelling Joe Schmoe rather than Joe Schmo, in part because it pluralizes better -- are actually one of the groups most responsible for me no longer having the ability to achieve gainful employment as a critic. But I don't begrudge them their right to do what they're doing -- not out loud, anyway. It's the out loud snobbery of statements like the one above that I find kind of shocking.

The above bit seems to be a direct dig at Rotten Tomatoes, but I'm starting to wonder if Rotten Tomatoes isn't more likable than its snooty rival. Rotten Tomatoes does indeed have 156 critic reviews that "count" for Boyhood, only two of which are rotten. In a system based on actual math, and using simple thumbs up-thumbs down metrics, those two negative reviews still leave Boyhood shy of that perfect 100. It carries a 99 freshness rating. (Interestingly, neither of the negative reviews belongs to Armond White. The two lonely contrarians in this case are a woman named Rebecca Cusey and a guy named Matt Pais.)

One assumes that Metacritic's 40 hallowed critics must be chuffed (to use an Australian term) that Metacritic considers them useful members of the critical community. But at the same time, eight of them must feel like their opinions don't mean a squirt of piss. Because there's certainly one thing that a perfect score draws attention to: the insignificance of the opinions of the naysayers. If a movie has a 93 or a 96 or something close to perfection, people aren't running any numbers to determine what Metacritic is actually saying. But if Metacritic says something has a perfect score and there are eight people -- that we know of -- that don't think it's perfect, one has to wonder why they even bother to post the reviews of those people. I mean, this is all about displaying only the content that conforms to Metacritic's preconceived notion of itself in the first place, right?

I'm starting to wonder if I don't have the dumbest of reasons for preferring Metacritic over Rotten Tomatoes, which is also a factor in why I like going to Mobil more than I like frequenting other gas stations. Simply put, I like Metacritic's fonts. I find its design aesthetically pleasing, and I do not find the Rotten Tomatoes design pleasing. In fact, I'd go so far as to say that Rotten Tomatoes looks cheap. Then again, maybe that makes sense -- a crisp, clean appearance for a snobby website, and something less fussy for an egalitarian democracy like RT.

As I write this post and find more and more things about Metacritic I don't like, I'm starting to view it as the kind of secretive organization that Kirby Dick railed against in This Film Is Not Yet Rated when he excoriated the MPAA. The only difference is that Metacritic is flippant about its secretive decisions, almost rubbing our noses in them, while the MPAA is just plain secretive. Metacritic, while being superficially more open by admitting its own secrecy, actually comes off worse.

Of course, none of this has any impact on my anticipation for seeing Boyhood, which, regardless of source, figures to land somewhere between a simply great film and The Best Film I Have Ever Seen. All I know for sure is that it's probably not perfect, and I don't know why Metacritic is so hell bent on telling us it is -- especially when they refuse to even tell us what their logic is in reaching that conclusion.

You'd think after writing all this I'll be ready to delete Metacritic from my favorites, but you'd be wrong. Fact is, there's still something about its snooty judgments that I crave. It speaks to my own inner snob, I guess.

Or, maybe I just like the fonts.