Models break down when you exploit them. Examples:

• “Number of lines of debugged code” is a good measure of programmer productivity, until you tie programmer reward to it. The programmers write very verbose code.
• Stock price is a good measure of the performace of a company, unless you tie management’s reward to it by giving them stock options. Then they manipulate the stock price, as happened in the ’90s at places like Enron and Worldcom.
• Macroeconomic models, for example that high unemployment coincided with low wage inflation and vice versa, are valid until people try to exploit them.
  

  Such trade-offs, [Robert Lucas] argued, existed only if no one expected policymakers to exploit them. Unanticipated inflation would erode the real value of wages, making workers cheaper to hire. But if central bankers tried to engineer such a result, by systematically loosening monetary policy, then forward-looking workers would pre-empt them, raising their wage claims in anticipation of higher inflation to come. Cheap money would result in higher prices, leaving unemployment unchanged.

  – The Economist July 13th 2006, Big questions and big numbers

What does this imply for adaptive behavior? When a creature discovers a relationship, that may hold as long as it doesn’t try to exploit it.