IT was called the London Diagonal, an enormous £1.5 billion tunnel that would whisk commuters under south London from Clapham to Oxford Street, solving at a stroke the chronic congestion on the capital's commuter lines.

The plan was put forward four years ago by Sea Containers, the Bermuda-quoted transport group, as part of a bid to take over South West trains, Britain's biggest rail franchise.

The turn of the century - and the one before it - was a time for swashbuckling rail entrepreneurs and ambitious grand projects.

A tunnel from Clapham wasn't enough for First Group, Sea Containers' rival bidder. It wanted one starting at Wimbledon. At the same time Sir Richard Branson was punting a plan to build a £3 billion high-speed line that would carry TGV-style trains up the east coast, drastically shortening journey times from London to Scotland.

Those days now seem far off. The collapse of Railtrack, the disastrous, multi-billion-pound cost overruns on the upgrade of the West Coast Main Line and the rapidly rising cost of running the existing network have put paid to the wilder flights of fancy that flourished in the early years of rail privatisation.

The industry is now firmly back under government control. The Treasury underwrites the borrowings of Network Rail, Railtrack's successor, while the Department for Transport has abolished the Strategic Rail Authority, and taken in-house the monitoring and control of train-operating companies. The Tory ideal of 'open access' with trains from different companies running on the same route and competing for passengers, has quietly been dropped on all but a few token routes.

Whitehall's influence is pervasive. Tender documents for rail franchises stipulate everything from timetables to train-cleaning regimes. Incumbent operators have been shown little respect, having in some cases been booted out for not paying enough attention to the detail of bids.

These days train companies do not even have the freedom to choose their own train; the specification for new rolling stock to replace the venerable High Speed Train, workhorse of British intercity lines, is being drawn up not by the companies who will buy or use them, but by a Department for Transport-led committee. Last week Whitehall flexed its muscles again, saying it would take away Branson's Cross Country franchise as it redrew the railway map.

Train bosses agree the swashbuckling days are gone, but say the government will shortly need rail entrepreneurs more than ever.

Rapidly rising passenger numbers, up 40% in the past decade and now at their highest level since the late 1950s, will require innovative thinking to cope with congestion and overcrowding, they say.

Branson said his appetite for the industry had not dimmed. "We will be bidding for the new franchise that will replace Cross Country, and we have some fairly interesting ideas to bring to the party. The last few years have not been easy, but we want to finish what we started. We are certainly not making much money out of railways at the moment � we see it as more of a public service."

Tony Collins, Virgin Trains' chief executive, said the government's intervention was a natural reaction to an industry that had swung out of control.

"It has swung like a pendulum. We had too little control after privatisation, some big problems, and now we have swung the other way. It will settle back down again."

On Virgin West Coast, (Virgin's other franchise, which it will have until 2012) passenger numbers are up 40% year-on-year, thanks to higher-frequency services. Collins said: "The government will eventually need private-sector investment in rolling stock and station enhancements to deal with this."

David Franks, chief executive of National Express's rail division, said the industry was now in a "back to basics" phase. "The government and the secretary of state are focused on recovery of the industry from the dreadful days post-Hatfield [the fatal derailment at Hatfield, which brought the network close to paralysis] and on cost control. Performance and cost are driving train companies at the moment."

Transport bosses have abandoned hopes of winning franchises with big-spending plans for new lines or tunnels.

Chris Moyes, chief executive of Go-Ahead, the quoted transport group, said mounting a bid was now a big exercise. "The average bid is a very elaborate and complex document. All bidders are now employing an army of advisers and devoting substantial resources to each franchise tender," he said.

Franks said that in the first round of bidding for rail franchises when British Rail was broken up in 1996, bidding costs were minimal. "The specifications and requirements in the bids now are far too great. I think the department realises this and is working with us to try to make some changes."

Bidders have to ensure that they are compliant with the department's tender documents, and then have the chance to show their entrepreneurial skills by coming up with ideas to make more money.

Go-Ahead is bidding for the Integrated Kent Franchise, which will combine commuter lines to the southeast of London with a new high-speed service on the Channel Tunnel Rail Link from Ashford to St Pancras. The winner will play a key role in providing transport to the Olympic Games in 2012.

Moyes said that while franchise bids were now prescriptive, innovative thinking on the part of bidders would play a part in the final decision. "We do not have a blank canvas, but we are able to show what we can do to make more money for the government," he said.

Rail-industry leaders believe the rapid growth in passenger numbers means that the government will at some stage have to return to the private sector for investment in infrastructure.

Despite all the bold plans four years ago only one major example of private investment in infrastructure was successful. Chiltern Trains, which runs trains from Marylebone, was able to set up a special-purpose funding company to channel private money into infrastructure improvement.

"The growth forecasts we are looking at are phenomenal," said Franks. "Motorway congestion and high fuel prices are driving people onto trains, and in some places the network is already operating at absolute capacity," he said.

Rail entrepreneurs could help by coming up with innovative schemes to squeeze more capacity out of the existing network, Franks said. "There are things you can do with longer trains, creative pricing mechanisms, and changes to signalling. But that will only take you so far."