There are at least two potential reasons for this. First, the financial derivatives they rated were incredibly complex and difficult to evaluate. They included not just bundles of home mortgages themselves (which would be complex enough), but also products that provided "insurance" against the possibility that these mortgages would default, packages of bundled mortgages separated into different risk tiers (or "tranches"), and so forth. The ratings agencies put together the most sophisticated economic models they could manage, given the data available, and then based their ratings on these models, which were released to the financial community. But investment bankers had a strong economic interest in getting the highest possible prices for those bonds that represented the lowest possible cost (translation: they wanted to get the highest possible ratings for their bonds from the agencies, while providing as little genuine "safety" as the ratings agencies would accept). So these seven-figure bankers constantly probed the agencies' rating models, which had been created by five-figure analysts, and frequently they did find loopholes. So the reliability of the ratings system was rapidly undermined, with catastrophic consequences for the world financial system.

Michael Lewis, in his book The Big Short, gives some great examples of how bankers were able to game the system, based on the agencies' ratings models. According to Lewis, for example, they easily figured out that the agencies

"didn't actually evaluate the individual home loans, or even look at them, before issuing a rating on a bond made up of these loans. What they had was a model that characterized the portfolio of mortgages in a loan based on their AVERAGE characteristics... To meet the rating agency's standards - to maximize the % of triple-A-rated bonds created from any given pool of loans - the average FICO score of borrowers [i.e., credit score] in the pool needed to be around 615. But the opportunity for the Wall Street firms lay in the fact that this average could be reached in a number of ways. A pool of loans composed of all 615 scores was far less likely to suffer huge losses than a pool composed of half 550 and half 680. A person with a 550 was virtually certain to default and should never have been lent money in the first place. But the hole in the rating agencies' models allowed such a loan to be made anyway, and then pooled with some high-FICO borrowers to serve as collateral for a AAA bond!"

But yet another reason the agencies' ratings might not have been as objective as they should have been goes to the heart of an issue that is central to how information is created and disseminated, and will likely become more and more important as more information is disseminated online. This has to do with something economists call "agency bias" or "agency costs." This term does not refer to the ratings agency, but to the bias inherent in any action in which one person is acting as an agent for another. The agent's natural bias is to act in his own self-interest, even though he is obligated to act in the interest of others - either the ones who hired him, or some other constituency. Agency bias is what professionals are supposed to be immune from. Doctors (at least under the US's current healthcare structure) have a natural economic self-interest to perform more tests on a patient than are really needed, and to keep a patient ill for longer than necessary. Lawyers have a natural economic self-interest to extend the lawsuit or other case that their client is involved in. The reason we call these people "professional" has largely to due with the fact that professionals are expected to ignore their own self-interest, and always to act as the unbiased agents of their clients, representing their clients' genuine interests as if they were their own. Whenever an agent doesn't put the interests of his client ahead of his own self interest, it creates "agency costs" and erodes the efficiency of the economic system.

Here's the problem with respect to the ratings agencies: They are paid for their services by the investment banks on the "sell side" of the financial community. But the ratings themselves are supposed to inform and benefit the buyers of these financial instruments, not the sellers. If the agencies had actually admitted that many of the financial instruments were simply too complex to be accurately "rated" as to their genuine risk to investors, then they would have had to turn down a whole boatload of highly profitable engagements from Wall Street's eager sellers. The agencies' own self-interest was to procure more and more assignments from the investment banking community, while the bankers were interested in procuring the agencies' ratings in order to sell more and more financial derivatives. This is classic "agency bias."

However, if you think about it, agency bias like this affects not just lawyers, doctors, and ratings agencies, but literally every enterprise that has employees. Employees themselves are the ultimate agents, each one honor-bound to act in the interests of his or her employer at all times, but tempted on a daily basis to serve their own self interest, as well. When a business pays commissions to employees for generating product sales, it is explicitly acknowledging that commissionable sales are in the business's interest. And as the financial meltdown demonstrated, sales commissions can often prove irresistibly lucrative. Many of the individual traders who participated in the run-up to the crisis knew full well that some of their deals were not very sound. After all, they themselves were the ones who were gaming the system in order to con the rating agencies into issuing higher ratings. But hey, if the bank didn't want these deals in the firs place, they wouldn't have agreed to pay such high commissions, right?

Moreover, agency costs can be expected to play a more and more important economic role (with significant consequences) as companies gradually "virtualize" their organizations, adding independent partners to do work formerly done by full-time employees, and contracting with outsiders to outsource an increasing amount of work that used to be considered more central to an enterprise's functions.

The increasing costs of agency bias are likely to make it even more important for companies to base their future success on earning and keeping the genuine trust of customers, employees, and other economic partners.