I hope I won’t be out of line if I point out that abandoning X on one machine, eschewing both X and the framebuffer on another, and spending a lot of time compiling on the third … seems to have had a rather positive effect on my productivity.
Which is ironic to me, since that last machine, the fastest one, is the only one left that does — or at least can — run an X-driven graphical environment. The others either don’t, by choice, or can’t, because the hardware is ancient.
And yet in spite of that, I get the strange feeling that I am actually getting more done these days that I did when I was using a “typical” desktop.
Maybe I should change that to “distracted by a typical desktop.” Yes, I’ll just say it out loud, instead of insinuating: The time I spend drifting around the Internet, looking at random tidbits of news or trivia, is much slimmer in a text-only/framebuffer-only environment, than in a graphical one.
Nothing is lost in the conversion, in case you’re wondering. If I want to read the news, I read the news with elinks, and then I go back to work. If I want to dig up information on how to fix something, or if I want to find a torrent for a new distro release, or if I want to check Wikipedia for a random (and hopefully trustworthy) fact, I can still do those things. I just seem less likely to be distracted — time spent looking for point A is a lot shorter, if I’m not drifting away to look at points B, C, and D.
It’s true that I still have the option of a full X desktop on at least one machine, and probably another, if I’m willing to search-and-destroy my troubles with X and the Silicon Motion driver. But so long as I’m reliant on X-less systems, I am enjoying a considerable improvement in focus.
The moral of the story: Find an old machine and fine-tune it to run without a graphical desktop. You’ll be surprised at how much more you get done. 
I’ve noticed the same thing. Something about the graphical environment sucks me into a prolonged period of idle browsing.
Maybe because it IS so effortless? When all I have to do is point and click, my experience is largely passive — exactly like sitting in front of the teevee with the remote, channel-surfing for hours on end. And there are so many bright and shiny things competing for my attention.
In textmode, I’m more actively engaged. Less distracted, more focused.