by Bud Young

Pumps & Systems, November 2007

Here is a proven methodology to use for your bad actor improvement efforts.

A reliability engineer or rotating equipment engineer with a major pump fleet in his care always has some pumps that are as reliable as pet rocks. The only mechanics who know where those pumps are located are the ones charged with lubrication replenishment and routine vibration surveys.

But other pumps seem to spend as much time in the shop as they do in service. For mechanics and operators, these pumps are "bad actors" that should be converted to boat anchors at the first available opportunity. Since these bad actors seriously impact the fleet Mean Time Between Failure (MTBF), chasing them down is the best way to improve MTBF and increase profitability.

For example, in a July 2004 article in Pumps & Systems, Heinz Bloch estimates the average true cost of a pump repair to be $15,000. Using this estimate, assume a typical refinery has 2,000 operating pumps. With a two-year MTBF, that refinery spends $15 million each year on pump repair. But improve that MTBF to ten years and the annual repair costs drop to $3 million per year - meaning $12 million per year is chased to the bottom line.

The following methodology is primarily assembled from experience gained in bad actor improvement efforts on API-610 pumps in various refineries. However, much of what was learned there applies to bad actor pump remediation in any situation.

A Team Effort

Assuming the chase is a one-person responsibility is an error. The responsible engineer, maintenance mechanic, operator, and seal manufacturer each have a different view of the problem that needs to be heard.

The plant rotating machinery engineer or machinery reliability engineer is the logical team leader because he is in a position to accomplish most of the necessary work by himself and has access to the required data. He should also be familiar with the necessary math and spreadsheet software and be able to convert those data into coherent formats.

The most important team member - although he doesn't have to attend team meetings - is the senior management sponsor. The team leader talks to operators and mechanics and needs the assistance of technicians to perform vibration studies and take field measurements of flows, pressures, and amperage. These people all have supervisors that need them to do something else at that time. The management sponsor, who understands how the annual operating cost is connected to the MTBF, must offer his support in all discussions with the relevant supervisor. At the conclusion of the study, he must approve the required expenditures to affect the recommended changes.

There is typically one pump mechanic on the site who the other mechanics turn to for help in solving pump problems. He must be on the team because his knowledge of the bad actor and his unique experience make him invaluable.

The team leader will also need assistance from operators. Different pumps may have different operators with the necessary experience, so this could be a "rotating" team member. However, it is imperative that no team conclusion is reached until the relevant operator gives his approval.

Most large users (such as refineries) have an alliance with a mechanical seal vendor. In return for the mechanical seal business at that site, the seal vendor offers supplies and services that include warehousing spares, a maintained site seal database, and assistance with new application and upgrade projects. Since a significant percentage of failures are usually mechanical seal failures, the site representative for the seal vendor is also a necessary member of the team.

It is not a necessity that all team members attend each meeting, but the five team members mentioned above are critical. Prior to starting these team meetings, however, the team leader can work independently and accomplish a great deal on his own.

Identification

Which pumps are bad actors? The team leader has MTBF numbers at his disposal that are the obvious starting point. This number by itself, however, does not necessarily give an accurate picture.

Excluding that rare user that evaluates MTBF based on actual operating hours, the fact is that a pump running 24/7 that has to be rebuilt once a year impacts MTBF more than a pump running 10 hours a week that needs to be rebuilt every two years. However, improving the continuous-duty pump from one year to three years will not help the MTBF as much as improving the intermittent-duty pump from two years to ten years - and the latter problem will most likely be easier to identify and less expensive to solve.

Every pump in the bottom 25 percent of the MTBF spread should be evaluated, giving proper weight to annual hours of operation and severity of service.  

Study the Files

After identifying the bad actors, look at the pump files: the pump data sheet, purchase order, the original performance curve and other available file data. The team leader will copy this data for his own files and refer to it several times during the study.

Calculate the actual Suction Specific Speed (Nss). Using the Nss number, calculate the minimum continuous flow. The pump manufacturer probably gave a minimum continuous flow number when he sold the pump. That number is based on stable or thermal conditions, however, not on reliability factors. The minimum continuous flow for reliable long-term operation is the number that is needed, and that number is a percentage of flow at maximum-diameter best efficiency point (BEP) adjusted for Nss.

Talk to the Techs

If regular vibration surveys are included as part of the preventative maintenance program, have the vibration technician evaluate the results of that survey. If there is a repeating problem, does he have a theory to go with it?

Talk to the Operators

The pump data sheet explains where the pump is supposed to be operating. But where is it actually operating at this time? The operators and/or the responsible process engineer might confess that while the pump may have been sold for 1,000-gpm, it consistently runs at 500-gpm or 1,500-gpm. The pump also might be handling a completely different viscosity, temperature, and/or specific gravity than the original design conditions.

How do the operators know the current flow? If the flow is metered, how confident are the operators in the meter's accuracy? Does the meter reading approximately correspond with the process engineer's mass-flow numbers for that loop? What are the suction and discharge pressures? Using the measured suction pressure and vapor pressure, what is the calculated NPSHA?  Is the NPSHA sufficient?

Check the Shop Records

What is being done to this pump when it is in the shop? What parts were actually replaced during the previous times it was repaired? What is the actual impeller diameter (and part number) installed in that pump today? How many vanes are in that impeller? Did the last mechanic that repaired that pump record his findings?

Check the pump shop records for recent changes. There could have been a major change, such as a different impeller, a different seal design, a seal flush plan change, or different metallurgy that may have eliminated the common failure mode.

If such a change is found, that pump should be removed from the bad actor list until the next repair. The situation should be reevaluated at that time. If the change did not seem to solve the problem, the pump is returned to the bad actor list.

Take a Look

The next operational step is to visit the site and observe the problem (opportunity). If this were a new installation, what would be different? Could those differences be responsible for the difficulties that this pump seems to be having?

Starting with an overall view for identification purposes, take several digital photos of the pump from all angles. Include the suction and discharge pipe spools from the pump to the first pipe support or anchor. If there is a minimum-flow bypass loop, take photos of it. These photos will be very useful later during the team study.

Unless it is a hot-service pump, alignment may not be a concern. Given all the times this pump has been removed and replaced, the mechanics have probably been very diligent with alignment and soft-foot. If it is a hot-service pump, however, there are other factors to consider.

Unless the pump is furnished with tooling balls (Essinger balls), it is probably being aligned only at ambient conditions. What happens to that alignment when the pump heats up?

If it were installed as a new pump today, the responsible engineer should have run the necessary stress calculations. He will have designed the pipe spools and pipe anchors to compensate for any thermal-induced movement. However, pipe stress analysis software wasn't in common use until fairly recently. If this pipe was installed under the old "cold-spring" rules, the piping system could well be contributing to the problem. If the failure mode is consistent with a twisted casing (bearings, seals, shafts), a piping engineer should run the piping stress isometrics through his computer program.

Another area to be checked with hot-service pumps is the warm-up procedure. How do operators warm up the pump prior to starting? Is there an installed spare that starts automatically? How is the spare pump kept hot-and-ready? Usually a small flow is kept back-flushing through the standby pump by means of a check valve bypass line or a small drilled hole in the check valve disc. If this is a top-suction, top-discharge pump, however, this method is not sufficient. In that case, the recirculation must be brought in through the bottom casing drain connection.

The piping support system should be analyzed. Are the hangers functioning correctly? Is that pipe simply hanging on the pump nozzle? Pump nozzles are capable of supporting massive loads. Loads in excess of the manufacturer's ratings can twist the casing to the point that MTBF is severely impacted.

With the pump running, press your finger against the joint between the grout and the baseplate. Then press your finger against the joint between the grout cap and the foundation. Is there any differential movement or is that machine still firmly attached to the foundation? Whether the existing grout is cementatious or epoxy is immaterial. What is crucial is a solid foundation. If the baseplate has broken loose from the grout, or the grout cap from the foundation, it is time for an epoxy re-grout.

Did the discussion with the operators reveal that the pump is operating a significant part of the time at less than minimum reliable flow? If so, is there a minimum-flow bypass loop installed? Are the bypass loop valves open? Is it a simple orifice loop, or a meter-controlled loop? If it is an orifice loop, is the orifice properly sized? Has the orifice "washed-out"?

If the minimum-flow bypass loop is meter-controlled, is the meter transmitter set to the proper value? Replacing an orifice loop with a meter-controlled loop is an improvement that rapidly pays for itself if the bypass quantity is significant.

If the operators do not know at which point the pump is operating on its performance curve, or if there is reason to suspect that their data are wrong, it is imperative to determine the actual conditions. A non-intrusive flowmeter with time recording capability is required. A technician trained to use that meter is necessary. Certified test pressure gauges with the proper ranges are also required. If motor amperage readings can be taken at the same time, a rough-check can also be made on the pump efficiency.

Do a spot-check on the operating point recording flow, suction pressure, and discharge pressure. If possible, include motor amperage. Leave the meter in place and get a recording of flow at 15-min intervals over a 24-hour period. The meter could be misinterpreted if only total flow divided by total minutes is considered. The pump could be running half of the time at 50 percent of BEP and half of the time at 150 percent of BEP. What is needed are the 96 data points recorded during that 24-hour span. It could well be that the pump is operating everywhere on its curve except the design point.

Is the pump environment having an impact? For example, there was a situation where a refinery had frequent failures with a long skid. The problem turned out to be the desert sun. As the day progressed, different parts of the skid were exposed to the direct sun and the alignment varied accordingly. The solution was to build a shade wall.

Heating and cooling the skid as a complete unit (such as the daily temperature change or a sudden cooling rainfall) is not usually a problem, provided that the temperature change affects the entire skid. Heating or cooling only part of the skid (sun, rain, steam exhaust, water drain) could cause misalignment.

Team Study - Identifying the Problem

It is rare that the root cause is obvious and that "guaranteed" solutions are apparent. Most recommendations are going to be the team's "best hypothesis," and not all hypotheses are going to be correct.

There will probably be some bad actors that will not be eliminated. In some cases, the root cause of the frequent failures will never be identified. In other cases, the solution, while known, cannot be economically justified. Most of the bad actors can be corrected, however, if the team leader does a complete investigation prior to convening with the team.

Your initial step is to lay out all of the site photographs and report to the team the results of the initial investigations. What do other team members see that the team leader missed during his observations? Is there a common failure mode on this particular pump? If so, what might be causing it?

If frequent seal and/or bearing replacement is the usual failure mode, could shaft flexibility be the cause? Many API-610 pumps manufactured prior to the 7th Edition (February, 1989) had high Shaft Flexibility Index (SFI) numbers. Shaft flexibility in the stuffing box area cannot be tolerated by today's low-emissions mechanical seals.

If the bad actor is an API-610 pump manufactured prior to that date, and the failure mode is consistent with a high SFI, consider a power frame upgrade to at least 7th Edition standards. These upgrades are available from several aftermarket sources as well as most OEMs. Such upgrades are far cheaper than replacing the complete pump, since the baseplate, foundation, and piping spools do not need to be replaced.

Be aware of bad designs. Some pump designs are considerably troublesome. They have been "outlawed" by API-610 and most OEMs no longer offer them. Double-suction overhung and two-stage overhung pumps are cases in point. Most of those pumps had unacceptable SFI numbers that could not be corrected by a simple power frame upgrade. Other designs, such as impellers with even-numbers of vanes in double-volute pumps, should be viewed critically. A vane-pass frequency equal to twice the running speed can lead to sympathetic vibration problems that might well be the cause of frequent bearing and/or seal failures.

Remember that frequent bearing problems could also simply be a lubrication problem. If the bad actor is one of a group of similar pumps in the same area, and the other pumps do not have the same problem, lubrication wouldn't be the number one suspect. If this pump is isolated by itself, and especially if it is in a dusty area, observe it more closely. Is there any way to get oil mist to that area? If not, is there a nitrogen header close to this pump site? Are there filtered vents and bearing closure seals on that machine?

Take a look at suction conditions. Does the pump have sufficient NPSHA? What's the suction piping configuration? Is there a satisfactory straight run upstream of the pump? Is that important on this pump type? What about the upstream elbow geometry? What did the field study show on the current condition of the suction piping anchors and supports?

Take a look at the discharge conditions. Does the pump have a minimum-flow bypass loop? Does it need one? Where on its performance curve is that pump actually operating today? What did the field study show on the current condition of the discharge piping anchors and supports?

Is there a recurring vibration problem with this pump? How accurately is the rotor being balanced during pump repairs? What does the field study on the grout bond reveal?

Is the pump rated as a bad actor because of frequent mechanical seal failures? Has the seal vendor identified the seal failure mode? Does the seal vendor have a suggestion on seal type, seal face materials, seal metallurgy, or seal flush plan that might cure the problem?

Economics

Understand that the "fixes" will be reviewed by the bean-counters before being approved. For this reason, keep your solutions as economical as possible. Your company is probably willing to spend $30,000 to save $15,000 every year. They probably will not approve the $300,000 solution.

Use Caution with "Obvious" Solutions

Do not assume that because the bad actor fails to follow all of the normally accepted rules, the rule deviation is responsible for the failures. A high Nss number, for example, is only a concern if the pump is working off its BEP. If the pump can be controlled to a flow near the BEP, that Nss number is relatively insignificant.

Violating the accepted rule of six to eight diameters of straight-run suction piping is also not necessarily significant. This rule is especially not relevant if there is a reducer at the pump suction nozzle, if it is a top-suction overhung pump, or if it is a vertical-in-line pump.

An obsolete design no longer acceptable to API would be suspect only if the failure mode is consistent with the design problem. For example, bearing and/or seal failure possibly caused by high SFI.

Pump Replacement

A common problem today is that existing plants are being expanded far beyond their original design criteria. Thus the pumps are required to deliver flow and/or pressure values beyond the capability of the original pump design. At the same time, they are not allowed an increase in NPSHA. It is also occasionally necessary to replace pumps because the old design simply cannot reliably hold today's low-emissions mechanical seals.

The team may initially want to recommend complete pump replacement. However, this should be an absolute last resort. It is the responsibility of plant engineering to make the existing equipment work if at all possible. Equipment cost is only a minor part of the replacement cost. There is also the cost of demolition of the existing installation, followed by the cost of new foundations, piping, etc. Construction contractor estimators advise that the total installed cost of a complete replacement pump is usually three to five times the equipment cost.

Impeller Replacement

At times a change in pump hydraulics is required beyond what can be accomplished with a different impeller diameter. Major hydraulic changes can occasionally be made by installing a different impeller in the existing pump (volute).

The OEM may have the impeller that is needed; some OEMs have several impellers for a given volute. If this is not the case, there are reputable aftermarket shops that can sometimes custom-design a new impeller for the existing pump.

A new impeller might lead to a larger motor size, but increasing motor horsepower due to new hydraulics is not necessarily a major problem. It is sometimes possible to double the motor horsepower without major changes to the existing baseplate and foundation. With a horsepower increase however, the plant electrical engineer must be consulted on available power at that location and on possible requirements for a new motor starter and cabling.

Custom Pump Replacement

If the pump design itself is deficient, there are pump manufacturers that will custom-build a pump to fit your existing base and piping interface points. Such manufacturers can custom-design the hydraulics to fit the operating conditions at the same time. Is this expensive? Absolutely. However, it is usually less expensive than demolishing and replacing the existing foundation and piping.

Add a Booster Pump

If the bad actor problem is due to insufficient NPSHA or insufficient head, and none of the above solutions are applicable, consider a low-speed, low NPSHR booster pump upstream of the bad actor. An engineer normally does not want two pumps in series because it introduces added failure points. A properly-applied 900-rpm or 1200-rpm centrifugal pump, however, will run a significant amount of time between failures. Adding a low-power vertical-in-line centrifugal pump is relatively inexpensive.

Ongoing MTBF Improvement

Chasing bad actors, while important, should not be the principal focus. The focus should be the MTBF number itself, and chasing bad actors is only one way to improve the number. Continue to push that number with all possible alternatives. No pump should leave the shop without bearing closure seals and filtered housing vents.

Except in rare temperature/fluid situations, every case/impeller re-ring should be with PEEK, Vespel, or similar composite materials. Every rebalance should be as precise as the balance machine allows. The seal vendor representative should be a familiar face in the pump shop. Whenever there is a new project in the area of existing pumps, try to include the necessary funding to initiate or expand your oil mist system.

Bud Young is a rotating machinery consultant. He can be contacted at 760-646-2433.