Software development: method driven or anything goes?

Anybody ever involved in real software development has learned that the usual software development methodologies seem to never comply with ones own experience. Nowadays - with the emergence of what I call "social" software development methodologies like XP and Scrum - this is not really news for us. But just about a year ago it looked like RUP would be the cure for all. This lecture makes a statement against high-ceremony software development processes and especially against the common belief among managers that you can "buy a success process" as the small book on RUP seems to suggest. Interestingly, in Objectspectrum 7/2001 Philipp Kruchten discusses so-called "misunderstandings" of RUP that turns RUP into a waterfall like process. He may be right but IMHO he also tries to schmuse into the now so popular XP, agile, scrum etc. movements (link to the fowler paper). But at least from what I read about RUP there is nothing in there that covers the fact that software construction is a deeply social process between highly skilled individuals. 30+ slides in German!.