Are we now India -- Electric Woes

[whighlander]

Today's Herald on the Back Bay Outage -- aka -- "The Back Bay Dark Pru"

Hynes convention canceled due to outage

By Greg Turner And Ira Kantor
Thursday, March 15, 2012 - Updated 41 minutes ago

The Back Bay power outage has forced the cancellation of a three-day gathering of a physical education professionals at the Hynes Convention Center.

The Alliance for Health, Physical Education, Recreation and Dance lost the first of three days booked at the Hynes for its 127th national convention after the neighborhood went dark following Tuesday night’s transformer fire.

Earlier today, the Virginia-based group scrapped the rest of the Hynes events. Some sessions that had been scheduled at the adjacent Sheraton hotel are still taking place, and a handful of Hynes events have been relocated, but the convention’s main exhibit hall with 300-plus booths never opened.

This afternoon Tom May, NSTAR, CEO held a press conference in front of the Neiman Marcus, essentially in the footprint of the new Tower at Copley

from NECN "

NStar CEO: 'The service is back now'

(NECN) - NStar's CEO held a news conference on Thursday announcing that power was back on for all of Boston except the Prudential Center.

"We understand the inconvenience that this has caused (Boston NStar customers), many of the businesses, the small businesses here," said NStar CEO Tom May. "I can assure them we have been working day and night. We've had over a thousand people on the street trying to get their service back, and in fact, the service is back now. We have reconnected our network."

The exception to the network is the Prudential Center, which NStar officials expect to have power again by the end of Thursday. "

Supreme irony -- Guess which large building houses Mr. May and the NSTAR Hq.?

times up

Its the Pru!!

BUT - MORE SERIOUSLY:
How prepared is Boston for major infrastructure failures?

In Bengeluru (Bangalore) India -- a large fenced in area behind the Pru would have held enough back-up generation to restore full power in minutes

BREAKING NEWS -- As NSTAR is restoring power to the Pru in blocks of 10 floors (each apparently has is own distribution transfomer) Emergency crews are now responding to 3 manhole fires on Huntington Ave behind the Pru and 101 and 111 Huntington

 

[BostonUrbEx]

I'd say that this is the beginning of perhaps many events across the country, despite the current woe-is-Boston attitude revolving around this entire event.

And I'd like to say that true competition, rather than legally contracted utility monopolization, would have likely prevented such an occurrence, but unfortunately I fear I do not fully understand the negative effects of having redundant (two or more companies with overlapping service areas) electrical grids -- if any. I believe in free markets and competition, but not blindly. I think we should dig in and look into giving up our utility contracts and letting the markets do their thing.

Can someone explain to me why utilities came under such stringent government controll in the first place? My understanding is 1. Companies eventually found themselves in a utility privately created monopoly, 2. The government stepped in and ran utilities themselves, 3. The government started contracting out utilities for private companies to run under their regulations. So can someone clarify if this is correct, and why points 1 and 2 happened?

 

[Tombstoner]

In Bengeluru (Bangalore) India -- a large fenced in area behind the Pru would have held enough back-up generation to restore full power in minutes

You've answered your own question: apparently we are not up to Indian standards.

 

[whighlander]

You've answered your own question: apparently we are not up to Indian standards.

Tomb -- Not Quite

Tom May said in his presser that he had not witnessed a similar failure of the downtown grid in his 35 years with Edison and NSTAR

In India -- the equivalent CEO might be able to say that he/she had not witnessed a similar failure in 48 hours or perhaps 4 days? But the common Indian level of unreliability has spurred the major consumers to make up for the unreliability of the utility grid with local generation.

What we should consider in a new district such as the SPID is a combination of the standard downtown grid of transformers, switches and interconnects along with an embedded web of local generation so that if needed the SPID as a whole could self power for a day or two

 

[HenryAlan]

I think the point about frequency breeding preparedness is important, as is the point that we are probably going to start seeing more of that frequency. We build to the very highest standard in this country, but we hugely short change maintenance. And it's not that we can't afford it. We choose this result by enriching a small minority at the expense of longer term stability.

 

[whighlander]

I think the point about frequency breeding preparedness is important, as is the point that we are probably going to start seeing more of that frequency. We build to the very highest standard in this country, but we hugely short change maintenance. And it's not that we can't afford it. We choose this result by enriching a small minority at the expense of longer term stability.

Henry -- that's a good point though apparently in this case it wouldn't matter

According to what I've heard so far -- the immediate cause of the outage was a failure of the connector between the oil-filled underground line and the transformer in the vault at 19 Scotia Street.

A reporter asked Mr. May about how often these fail -- he replied that in 35 years in the business that he'd never seen this failure. When the reporter asked about inspections -- he replied that no inspection even minutes before the failure would have detected anything -- And I suspect that he's telling it like it is.

However, just because this type of failure is so damaging though rare -- this type of failure has to be included in the failure analysis and recovery planning process.

 

[F-Line to Dudley]

Get used to it. Grid failures are going to be the "exploding mid-19th century rusted-out water mains" of the 21st century urban core. Underground and overhead high-tension power lines are some of the most obscure critical infrastructure in terms of public consciousness, so the investment in basic maintenance has been nil compared to, say, road bridges where people will easily bark up the chain about the rust they're seeing and bumpy ride. You don't know the grid is on the fritz until it gives out. Severely.

I think we're a few 2004 NYC blackouts away from our leaders getting the message. Stuff like NStar's and CL&P's (in the post-Irene and October storm debacles) sluggishness at repairs is seen rightly as monopolistic corporate dysfunction, but that masks to some degree how overwhelmed their qualified techs are out in the field in an emergency knowing the true fragile state of their infrastructure and knowing what they don't know about all that can go wrong with the unmapped spaghetti mess of wiring. We saw that unknown at work this afternoon after the juice went on when manholes started exploding all over Back Bay.


The "good" thing is that a mass blackout socks every income and demographic and political group equally, and makes them equally cranky. So there is hell to pay in a systemic failure. But there'll be no prevention. There'll have to be lost weeks of productivity, calculable billions in lost regional revenue, local/national/international embarrassment, hearings, citizens screaming at harried pols, sacrificial lambs, and all that...repeatedly...before that most-obscure of infrastructure takes center stage.


In other words...move to Needham. Municipal power supply FTW!

 

[Commuting Boston Student]

I'd say that this is the beginning of perhaps many events across the country, despite the current woe-is-Boston attitude revolving around this entire event.

And I'd like to say that true competition, rather than legally contracted utility monopolization, would have likely prevented such an occurrence, but unfortunately I fear I do not fully understand the negative effects of having redundant (two or more companies with overlapping service areas) electrical grids -- if any. I believe in free markets and competition, but not blindly. I think we should dig in and look into giving up our utility contracts and letting the markets do their thing.

Can someone explain to me why utilities came under such stringent government controll in the first place? My understanding is 1. Companies eventually found themselves in a utility privately created monopoly, 2. The government stepped in and ran utilities themselves, 3. The government started contracting out utilities for private companies to run under their regulations. So can someone clarify if this is correct, and why points 1 and 2 happened?

There's only so much space in the world you can dedicate to infrastructure like power cables, water lines, et cetera. If you let the free market into the utility business, it is not like going to the store where you can walk down an aisle and pick the item off the shelf that you like best. All the newly competing, say, electricity companies will need their own separate lines, cables, and so on, and so forth. Once you get out of the city, you can sometimes see overhead power lines strung to and fro - imagine taking those lines and having the exact same lines on top of one another four or five times for your four or five competing electric companies.

It's just a great way to get into a big mess very quickly.

 

[whighlander]

Get used to it. Grid failures are going to be the "exploding mid-19th century rusted-out water mains" of the 21st century urban core. Underground and overhead high-tension power lines are some of the most obscure critical infrastructure in terms of public consciousness, so the investment in basic maintenance has been nil compared to, say, road bridges where people will easily bark up the chain about the rust they're seeing and bumpy ride. You don't know the grid is on the fritz until it gives out. Severely.

I think we're a few 2004 NYC blackouts away from our leaders getting the message. Stuff like NStar's and CL&P's (in the post-Irene and October storm debacles) sluggishness at repairs is seen rightly as monopolistic corporate dysfunction, but that masks to some degree how overwhelmed their qualified techs are out in the field in an emergency knowing the true fragile state of their infrastructure and knowing what they don't know about all that can go wrong with the unmapped spaghetti mess of wiring. We saw that unknown at work this afternoon after the juice went on when manholes started exploding all over Back Bay.


The "good" thing is that a mass blackout socks every income and demographic and political group equally, and makes them equally cranky. So there is hell to pay in a systemic failure. But there'll be no prevention. There'll have to be lost weeks of productivity, calculable billions in lost regional revenue, local/national/international embarrassment, hearings, citizens screaming at harried pols, sacrificial lambs, and all that...repeatedly...before that most-obscure of infrastructure takes center stage.


In other words...move to Needham. Municipal power supply FTW!

F-Line -- while searching for some information on the Scotia Street transformer I googled a pdf appendix to the proposal to ISO New England about the underground 345 kV cable network tying major generation or above ground HV transmission stations together and feeding the Boston underground 115 kV network.

The document had a wiring diagram which included Scotia St., Carver St. and a couple of the other substations mentioned by Tom May in his presser yesterday. Contrary to what you said above --

Underground and overhead high-tension power lines... investment in basic maintenance has been nil compared to, say, road bridges...You don't know the grid is on the fritz until it gives out. Severely...NStar's overwhelmed their qualified techs are out in the field in an emergency knowing the true fragile state of their infrastructure and knowing what they don't know about all that can go wrong with the unmapped spaghetti mess of wiring

Actually -- NSTAR's underground 115 KV grid is very solid -- there might be some issue about the size of the feeder transformers for large complexes such as the Pru where each group of 10 floors has its own feeder.

By comparison to the tree and branches in Needham or any suburb there is a lot more redundancy in Boston by the Pru. There were two separate 115 kV lines into Scotia from 2 different directions and two transformers feeding the distribution bus at 14 KV. However, unfortunately, failure in the connector for one of the 115 kV lines ignited the oil in the line -- that was the thick black smoke. The fire suppression system worked perfectly and poured water down from above -- a process which saved the transformer but necessitated blowing a bunch of breakers to cut Scotia St. out of the grid. This was the only single point failure in the system and like a wing separating from a fuselage there is no feasible redundancy -- my only complaint was that NSTAR and major customers such as the Sheraton, Hynes and Pru didn't work together to design and build a local "generator park" separate from the Scotia transformers and able to handle the full load as long as there was fuel.


By the way a few manholes were popping on Huntington Ave., due to transient overloads -- just like what happens when you flip your main breaker - some of the branches might be transiently overloaded by starting surges of some loads (e.g. motors starting, etc.) -- so some of the feeder breakers in the manholes blew. As Tom May said to put in place the temporary back-up network getting feeds from generators and more than 5 miles of temporary wiring -- over 1000 separate opening and closing of breakers was required. -- more will be required this weekend as the temporary network is transitioned to an interim permanent network and then some additional tweaking as all of the functions of Scotia St. are restored.

 

[bbfen]

F-Line -- while searching for some information on the Scotia Street transformer I googled a pdf appendix to the proposal to ISO New England about the underground 345 kV cable network tying major generation or above ground HV transmission stations together and feeding the Boston underground 115 kV network.

The document had a wiring diagram which included Scotia St., Carver St. and a couple of the other substations mentioned by Tom May in his presser yesterday. Contrary to what you said above --

Actually -- NSTAR's underground 115 KV grid is very solid -- there might be some issue about the size of the feeder transformers for large complexes such as the Pru where each group of 10 floors has its own feeder.

By comparison to the tree and branches in Needham or any suburb there is a lot more redundancy in Boston by the Pru. There were two separate 115 kV lines into Scotia from 2 different directions and two transformers feeding the distribution bus at 14 KV. However, unfortunately, failure in the connector for one of the 115 kV lines ignited the oil in the line -- that was the thick black smoke. The fire suppression system worked perfectly and poured water down from above -- a process which saved the transformer but necessitated blowing a bunch of breakers to cut Scotia St. out of the grid. This was the only single point failure in the system and like a wing separating from a fuselage there is no feasible redundancy -- my only complaint was that NSTAR and major customers such as the Sheraton, Hynes and Pru didn't work together to design and build a local "generator park" separate from the Scotia transformers and able to handle the full load as long as there was fuel.


By the way a few manholes were popping on Huntington Ave., due to transient overloads -- just like what happens when you flip your main breaker - some of the branches might be transiently overloaded by starting surges of some loads (e.g. motors starting, etc.) -- so some of the feeder breakers in the manholes blew. As Tom May said to put in place the temporary back-up network getting feeds from generators and more than 5 miles of temporary wiring -- over 1000 separate opening and closing of breakers was required. -- more will be required this weekend as the temporary network is transitioned to an interim permanent network and then some additional tweaking as all of the functions of Scotia St. are restored.

Sort of.

1) NStar was delinquent beyond belief in responding to the initial BFD pleas to arrive at the substation.
2) Due to that, someone (from NStar? or their idiot second shift subs?) shut off much, much more than they needed to.
3) NStar bills all the businesses on Huntington Ave, so how could they not anticipate the load when they switched from gennies to in-ground power?
4) Tom May has no credibility. He allowed his company to lie to the city, public, and press* this week, yet there've been "off the record" acknowledgements that NStar has had no intention or impetus in meeting the deadlines promised by their PR flacks.

* By this, I'm referring to Thursday's declarations that all service was restored except for the Prudential tower. I missed the meeting where Fenway/Kenmore, Fenway East and the Fensgate neighborhood (Mass Ave to the Fenway, south of Boylston/north of Huntington) declared independence from the City of Boston. I definitely noticed that there was no power there, though.

 

[BostonUrbEx]

I'm hearing the entire town of Nahant is dark right now, no power.

 

[whighlander]

Sort of.

1) NStar was delinquent beyond belief in responding to the initial BFD pleas to arrive at the substation.
2) Due to that, someone (from NStar? or their idiot second shift subs?) shut off much, much more than they needed to.
3) NStar bills all the businesses on Huntington Ave, so how could they not anticipate the load when they switched from gennies to in-ground power?
4) Tom May has no credibility. He allowed his company to lie to the city, public, and press* this week, yet there've been "off the record" acknowledgements that NStar has had no intention or impetus in meeting the deadlines promised by their PR flacks.

* By this, I'm referring to Thursday's declarations that all service was restored except for the Prudential tower. I missed the meeting where Fenway/Kenmore, Fenway East and the Fensgate neighborhood (Mass Ave to the Fenway, south of Boylston/north of Huntington) declared independence from the City of Boston. I definitely noticed that there was no power there, though.

BBF -- maybe you missed an earlier post:

No -- the stuff that was shut was prudent as the initial scale of the damage was unknown

Switching high power circuits -- is a bit of an art -- there are currents which can exceed lightning strikes due to switching transients in some circumstances -- this is particularly true in the case of oil-filled underground transmission lines where there is quite a bit of energy stored in the electric field inside the insulating oil

NSTAR's HQ is in the Pru -- if anyone was to know it was dark it would be Tom May -- if the BFD didn't know how to call the NSTAR Hq-- well that's another problem

You might have also missed the point about the need to conduct over 1000 individual switching operations to put in place the surface wired temporary network and then about the same to get back on the underground network

Note if you are in the neighborhood there will be some planned outages tonight and possibly tomorrow night / early morning as the restoration continues

 

[F-Line to Dudley]

F-Line -- while searching for some information on the Scotia Street transformer I googled a pdf appendix to the proposal to ISO New England about the underground 345 kV cable network tying major generation or above ground HV transmission stations together and feeding the Boston underground 115 kV network.

The document had a wiring diagram which included Scotia St., Carver St. and a couple of the other substations mentioned by Tom May in his presser yesterday. Contrary to what you said above --

Actually -- NSTAR's underground 115 KV grid is very solid -- there might be some issue about the size of the feeder transformers for large complexes such as the Pru where each group of 10 floors has its own feeder.

By comparison to the tree and branches in Needham or any suburb there is a lot more redundancy in Boston by the Pru. There were two separate 115 kV lines into Scotia from 2 different directions and two transformers feeding the distribution bus at 14 KV. However, unfortunately, failure in the connector for one of the 115 kV lines ignited the oil in the line -- that was the thick black smoke. The fire suppression system worked perfectly and poured water down from above -- a process which saved the transformer but necessitated blowing a bunch of breakers to cut Scotia St. out of the grid. This was the only single point failure in the system and like a wing separating from a fuselage there is no feasible redundancy -- my only complaint was that NSTAR and major customers such as the Sheraton, Hynes and Pru didn't work together to design and build a local "generator park" separate from the Scotia transformers and able to handle the full load as long as there was fuel.


By the way a few manholes were popping on Huntington Ave., due to transient overloads -- just like what happens when you flip your main breaker - some of the branches might be transiently overloaded by starting surges of some loads (e.g. motors starting, etc.) -- so some of the feeder breakers in the manholes blew. As Tom May said to put in place the temporary back-up network getting feeds from generators and more than 5 miles of temporary wiring -- over 1000 separate opening and closing of breakers was required. -- more will be required this weekend as the temporary network is transitioned to an interim permanent network and then some additional tweaking as all of the functions of Scotia St. are restored.

The Boston grid power has to come from somewhere. Quebec mostly, touching down in MA in Ayer then fed through to the rest of the region. THOSE are the deferred-maintenance lines that are a ticking time bomb. I have no doubt that NStar's been keeping up in Back Bay because of the scale of development in the area. That's revenue for them. It's the ultra-high capacity regional transmission lines that have no obvious effect on stock price that the utility conglomerates are letting decay to spit as they squeeze profit margins. It's been well-known that the fault tolerance on the Eastern Interconnection was old and inadequate since 1989 when a solar storm melted Hydro Quebec's transformers. That HVDC interconnection is also the one that chain-reacted in the 2004 NYC blackout. No amount of local generating save for isolated municipal grids like Needham can stop a wide-scale blackout when an interconnection of that scale fails, save for isolating the chain reaction to an entire region (NYC) before it spreads like the '65 blackout. Nothing is going to provide juice to the cities in a systemic failure like that, and NStar most definitely has to worry about creaky transmission lines criscrossing the state out of that Ayer touchdown asploding local substations in a catastrophic fault.

Only Congress has the heft to appropriate the funds to shore up maintenance on that kind of scale, and it's megaproject-level so of course they're not doing it. This is where we're falling behind other countries that have been keeping up with this. And it goes hand-in-hand with maintenance on power generation. Building and renewing power plants for efficiency leads to building and renewing HVDC lines for efficiency. Natural resource-poorer countries in Europe have been doing this on wide scale for the last 20 years in addition to the prime developing countries (India, China, better-off Middle Eastern countries). Same private penny-pinching has left our energy generation flat for decades. The only place in the U.S. where a major new transmission interconnection is being built is in a swath from New Mexico to Texas, where the power speculation market is still hot. There isn't a single interconnection sharing the power regions east of the Mississippi except that fragile lifeline in Quebec that Hydro Quebec is gambling on deferred maintenance: http://upload.wikimedia.org/wikipedia/commons/f/f4/NERC-map-en.svg.


Get used to a rapidly increasing rate of failure. Both the huge blackouts once or twice a decade and major local failures on the regional utilities' plug-ins to that trunk. This is the next industry where we're going to get a sore lesson in the implications of over-privatization of a public resource.

 

[choo]

new england only gets about 13% of its power from Quebec, a non-trivial amount but it is not totally dependent. Additionally the Maine Power Reliability Project upgrading 345kV lines in Maine and the New England East West Solution in Southern New England represent billions in current projects under construction. And with New England well above its NERC mandated reserve margin, I think the talk of impending regional blackouts and brownouts is an overblown scare tactic.

Now, can NStar and all the utilities do better to bring power back on after failure, yes! of course. But often the people writing and on TV and commenting take a very simplistic view and have no understanding about how a power system operates. There is redundancy in the system, but you can't just go around building extra underutilized substations at $20 million a pop. A single HVDC substation can easily be more than $200 million.

As for the competition argument. Massachusetts is a deregulated market, you can sign up for a number of other energy providers this afternoon if thats how you want to spend your St. Patricks day, but it isn't going to change the infrastructure. NStar, National Grid and Northeast Utilities are regulated utilities. They get a 9%-11% ROI on their operations- so they actually have the incentive to do more work to capture more money in absolute terms. But they have to go to the regulators and get the ratepayers to pay for it- and even though electricity is ridiculously cheap, ratepayers do not want to spend to make the necessary investments. Most prefer to just complain when something goes wrong.

(Note: I do not work for a utility )

 

[Digital_Islandboy]

Are authorities sure--- the solar flares didn't cause it? I have the free app 3D sun ( http://3dsun.org/ ) and the sun has been throwing off many ray burts towards Earth lately.

 

[KentXie]

And I'd like to say that true competition, rather than legally contracted utility monopolization, would have likely prevented such an occurrence, but unfortunately I fear I do not fully understand the negative effects of having redundant (two or more companies with overlapping service areas) electrical grids -- if any. I believe in free markets and competition, but not blindly. I think we should dig in and look into giving up our utility contracts and letting the markets do their thing.

Can someone explain to me why utilities came under such stringent government controll in the first place? My understanding is 1. Companies eventually found themselves in a utility privately created monopoly, 2. The government stepped in and ran utilities themselves, 3. The government started contracting out utilities for private companies to run under their regulations. So can someone clarify if this is correct, and why points 1 and 2 happened?

I can answer your question. Running a utility company is expensive and having multiple small utility companies competing is not feasible. Picture this, imagine if there are 4 utility companies competing in Boston. Each company would needs its own utility equipments (wires, poles, transformer, etc.). They would also have to compete with each other for power plants. The cost would be enormous and would require passing a huge cost onto the customers and at prices that only a few can afford and thus not enough to make up for the cost.

This is why utility companies exist as natural monopolies.

Here's a wikipedia entry if you are still confused:

http://en.wikipedia.org/wiki/Natural_monopoly

But since they are still a monopoly, the government steps in to make sure that they do not gouge prices. Thus the regulation.

 

[Kahta]

I can answer your question. Running a utility company is expensive and having multiple small utility companies competing is not feasible. Picture this, imagine if there are 4 utility companies competing in Boston. Each company would needs its own utility equipments (wires, poles, transformer, etc.). They would also have to compete with each other for power plants. The cost would be enormous and would require passing a huge cost onto the customers and at prices that only a few can afford and thus not enough to make up for the cost.

This is why utility companies exist as natural monopolies.

Here's a wikipedia entry if you are still confused:

http://en.wikipedia.org/wiki/Natural_monopoly

But since they are still a monopoly, the government steps in to make sure that they do not gouge prices. Thus the regulation.

Or turn electricty lines into something similar to landlines where the service and the infrastructure and separate and the end user can choose who to buy their electricity from.

 

[choo]

Or turn electricty lines into something similar to landlines where the service and the infrastructure and separate and the end user can choose who to buy their electricity from.

Massachusetts is a deregulated electricity market. So you can do that very easily. And you probably should b/c its cheaper.

 

[KentXie]

Or turn electricty lines into something similar to landlines where the service and the infrastructure and separate and the end user can choose who to buy their electricity from.

You can choose from which electric company you want to buy from, just not the utility company. A few known ones are ConEdison, Hess, Sempra, Constellation New Energy etc.

 

[Kahta]

Massachusetts is a deregulated electricity market. So you can do that very easily. And you probably should b/c its cheaper.

I'm from Groton (municipal utility) and when I lived in Brookline my utilities were all packaged into my rent.