Monday, March 16, 2009

SAPPU

SAP is a huge, mysterious, expensive animal.

In my very private opinion it is probably the worst ERP system you can buy today. Hence, most whiteshirts will choose it.

To compensate for the fact that it's old and silly technology, it's also exceedingly expensive. Introducing SAP to your company is the only reliable way to tell whether your company is so financially strong that it almost resembles a monopoly.

But what I really hate about SAP is that it removes people from the Oracle database field. I think most of us have experienced the following scenario:

A colleague or a bunch of colleagues are selected to help implement SAP. Until then they've been ordinary DBA's, fixing stuff, running databases and leading normal family lives.

Then they go away for EXTENSIVE training over a LONG period of time. In between the 42 week-long classes they have to take (per year, of course), they usually rest with their families and might show up for short, social functions among their (still) colleagues. But they have these myserious, far-away eyes... you can't quite reach them.

Then you get the famous message by mouth or email stating:

"We're now almost ready to , so for the next transition period of , I'll be working half of my time with my old stuff and half of my time with SAP before moving to full-time SAP obligations."

AND THAT'S THE LAST YOU EVER HEAR FROM THEM!

I think, but I could be wrong, that they're sucked into a place and time in space that the rest of us can't see or in other ways sense.

A Harry Potter-like parallel universe.

From which, mind you, they never return.

Numerous are the good Oracle DBA's who disappear from the real Oracle world this way.

Now it appears that in their parallel universe (the SAPPU, it could be called) they're slowly corrupted into thinking about Oracle databases the way the real (the few, the remaining) Oracle DBA's thought about databases in the 80's.

They're forced to unlearn all the right things they had learned. A kind of communist re-schooling or indoctrination. So sad.

Here follows a real mail thread from some friends of mine that know more about Oracle than most. They shall, of course, remain nameless - we still have no idea about the powers and general abilities of our disappeared friends in the SAPPU, and so a certain degree of fear for the unknown make us cautious....

======================================================================
Oracle Person One said:

I find myself once again embroiled in an SAP/Oracle Issue.

It seems I broke something in the production database by applying security patches, even though the 5 development and production servers have no such issues. (No, we can't afford a full system to test patches on - sigh...)

There are a number of ORA-7445 and ORA-600 errors being issued when DML is attempted, along with requisite trace files. (Yes, there is a an active SR already)

The SAP Basis team see's ORA-3113 and ORA-3114 errors.

I think I finally have them convinced to stop trying to 'fix' the ORA-3113...

Which brings me to the point: The usual method of troubleshooting SAP problems, as practiced by the SAP team, and nearly every SAP person I have worked with (not a terribly large sample - maybe I am just really lucky), goes something like this:

1. Search the SAP support site for every note containing ORA-600, ORA-7445. There are a few as you can well imagine.

2. Call a meeting to discuss which of the actions in these notes should be taken. Whether or not the contents of the note actually match the problem at hand seems to be irrelevant.

3. Ask the DBA to run a script (recommended by SAP support) to check for some problem or another in the data. This script will launch full table scans for every table in the database... Using a for/next loop. Fortunately for me, the script is broken as is. ... and I can't seem to find the problem with it.

4. Rinse and repeat - effectiveness is unimportant, only looking busy is important.

To memorialize this method, I have created a link to the following short, but accurately portrayed method of this troubleshooting methodology:

http://tinyurl.com/sap-troubleshooting-method
===============================================================
For that Person Two commented:

Make that, 'development and test'
===============================================================
Person Three need this off his chest:

So, regarding recent security patches that might cause havoc:

The one that prevents old client retries in clear text when the encrypted handshake is rejected affects some connection attempts.

“Repairing” the permissions of the archive log destination can ultimately get the archiver stuck far enough behind to toss a 7445 (I think) I didn’t look it up and since my databases never have problems I get rusty on the error messages. (tongue firmly in cheek). If memory serves (see previous sentence) one of the security patches suggests repairing the permissions on the archive log directory without telling you to make sure the ownership is correct.

What are your vintages? I’ve only done SAP stuff ONCE (and walked away quickly and quietly having proved that a certain physical reordering solved all their stated performance issues on the load testing system only to be informed that any manipulation of the data outside of SAP was not allowed.)
===============================================================
Person One felt he finally had someone to talk to who understood him:

Yup, that is SAP SOP.

So far I have refused to do that particular operation, except in a couple cases where it retrieved a lot of empty space due to archival of data.

The 'make work' analogy from "The Longest Yard" (the old one with Burt Reynolds and Eddie Albert) was used in reference to the "SAP Reorg" mentality when they last asked me to do a db reorg.

They didn't get it.

For those of you that don't already know it, the 'make work' for the prison workers in the facility where Reynold's character was incarcerated consisted of the ollowing:

Morning: shovel mud out of the swamp.
Afternoon: shovel mud back into the swamp.

By the classic definition of work, nothing was accomplished in the end.
But the inmates were still sweaty, tired, thirsty and hungry.
===============================================================
Person Five then finally could say this to a friendly crowd:

I wholeheartedly agree with both of you.

I spent almost 4 years as part of team that supported SAP. There were 3 dba's on the team, and both of the other two received their primary dba training from SAP. Their method was to manage EVERY database as if it were an SAP database.

I volunteered to do every non-SAP upgrade and rearranged everything back Oracle standards when they weren't looking. :) Took about 2 years to get to them all but it was well worth the trouble.

As for the SAP databases, patch application and upgrades always had different results on dev, test or prod. It was utterly baffling.
===============================================================

Ah well, based on all this, it is really no wonder that SAP is so wildly popular and has won the whole upper(ERP market) over less complicated, cheaper, more technologically advanced - and way more agile - competitors.

If you can sit in the local CEO club and claim: "We actually managed to pay for the WHOLE SAP implementation with our own money and we're still functioning in several departsments..." your fellow CEO's will know that you have more money than God or AIG's dealers.

You will have the uttermost respect from them, as they scramble to try and explain why their predecessors in their respective companies chose to implement something different from SAP. Cheaper, of course.

But one day, when they have gained enough financial strength....

Here's one final salute to all those lost colleagues from the Oracle database space. We'll always remember you. You'll never be forgotten. Long live your memory.

Wherever you are. In whatever lifeform.

37 Comments:

Anonymous Anonymous said...

I've never actually met a SAP admin, they have always been located elsewhere another site or office. Always though there is the impression that as a mere oracle DBA I wouldn't understand what they do, however it appears what i suspected is true, all I need to be a SAP admin is pretend its 1992 and check that buffer cache hit ratio.

4:06 AM  
Blogger Moans Nogood said...

Yes, the BCHR was actually used heavily by SAP's silly Early Watch script in the 90's when I was looking at performance problems at a major Danish brewery.

I really tried to tell Walldorf (that's where the support guys of SAP are) that it was wrong, wrong, wrong of them to keep insisting on increasing the BCHR, but they of course insisted that if we didn't follow their instructions we were not supported anymore.

I wonder how far they've come since w.r.t. optimisation technology and methods - now that even Oracle gave up any BCHR nonsens a decade ago or more :-).

5:24 AM  
Blogger Niall said...

hey mogens

I'm not sure it's true that oracle gave up BCHR a decade or so ago, certainly not in the space that's relevant here - the ERP thing. core people at oracle certainly got it then, and a few others outside :(, but it took a while to get that stuff out of oracle products properly, probably more than 5 years, since they were a bit distracted buying everyone in that timescale. SAP would have done well to react until recently, and even then what you get is rather dependent upon consultant quality.

I've seen a few SAP sites now, and to be honest it looks like you probably can do mostly the right thing, but you have to first know what you are doing and second be prepared to jump through a few hoops to do it.

so walldorf probably did get it a while ago, but for that to translate to actual customers (probably running circa 2005/6 installs right now) will take a while.

3:15 PM  
Blogger Jared said...

Much like at Oracle, the core database folks at SAP do know how the database works. There are people there that are quite good, but like Oracle, you have to get past the tiers of support to talk to these people.

While I have managed to do so at Oracle (A couple years ago I even got a phone call from an Oracle Developer to clear up an issue), I have never had this kind of access at SAP.

There is evidence of them however.
If you have access to the SAP Support Portal, look up note SAP Note "825653 - Oracle: Common misconceptions".

Among other things, it exposes the myth of the BCHR.

"12. The higher the hit rate of the Oracle buffer pool, the better the
database performance."

This includes an explanation of why this is not so.

There are currently 61 such gems in this note. I haven't yet finished reading it myself.

11:28 AM  
Blogger Moans Nogood said...

Here's another wonderful message from a friend:

There is a large retail chain of jewelry stores here in the US called the "Shane Company". It is one of those companies whose founder has differentiated his company by taping his own ubiquitous radio commercials himself. He has a nasal annoying voice, but of course we're all aware of the old adage that "there is no such thing as bad publicity". Recently, he's tried to move his on-air persona from "intense nerd" to "cool", and it is astonishingly (and unintentionally) funny. I've heard their radio commercials in Colorado, California, and a few other states. They seemed like a healthy, growing company.

Recently, the Shane Company went bankrupt. The nasal company founder has insisted that the company's "lifetime guarantee" on jewelry is still valid, and the commercials still air, but I hear them much less frequently now. For all I know, they're out of business now.

I recently heard from an ex-colleague, a former DBA manager and former DBA who now sells remote database administration services, that it was an SAP implementation that ballooned in cost from US$12m to over US$40m in costs that put the multi-generation family business into bankruptcy. Could be true, could be untrue. But it sounds true.

7:35 AM  
Anonymous Linda said...

Apparently the Shane Company rumor (i.e. blaming SAP) is true: link.

10:24 AM  
Anonymous dylan (has various blogs of low relevance for the DBA community) said...

Hello,

liked your blog and your blog-nickname, funny Danish humour. A friend sent me this, because I do SAP support (for our implementation, not employed by SAP myself), here's an excerpt from my reply, minus any private jokes and swearing you'll be glad to hear:

"...if I already had a working Oracle or anyother DB underpinning my small to mid-size business, you're right i would not be considering an SAP implementation. I would probably buy something like Basware for doing the invoices and Finance processing, and save myself a fortune.

However one proviso: if i was running a mid-size to large manufacturing business, whether the product was bottles of mould or getting oil out of sand in Canada, i would still definitely get SAP. Because it's the only ERP system that can handle Production Planning, Materials Management, Sales & Distribution, and Finance, in one integrated product. And the savings when this solution is working are vast. That is the secret of SAP's success. Oracle ERP solution was never really up to the job so lost out despite the better DB integration. As the guy [i.e. Moans Nogood i.e. the blogger whose site this is etc] realises, the people who make the business decisions are not that concerned about the extra SQL layer..." and the drugery of trying to make techie stuff work.

Hope that's not too defensive.

He's just sent me the oracle.china blog link too, also very funny.

5:17 AM  
Blogger Noons said...

Similar to the SAPPU, there is also the PSPU (PS as in PeopleSoft).

Here, initial costs are much lower.
But performance is always ratshit, just like with SAPPU.

And disk space usage is orders of magnitude higher than anything else.

Things are somewhat more linear:

ANY performance problem - and I mean ANY - is ALWAYS the direct result of the (incompetent) DBA not knowing about the undocumented parameter _make_PS_run_fast, which is invariably left at FALSE by said DBA. No one else knows where the parameter is so it really doesn't matter in the end.

ANY performance problem is to be investigated by first taking a COMPLETE copy of production into the development system. All 28000 tables and 35000 indexes. Of which normally only 300 are in use anyway, but who's counting?

ANY PSPU developer is always an expert at creating the longest possible SQL statement for the simplest possible problem.
It's always the responsibility of the DBA to then fix the horrendous performance caused by multiple merge join cartesians, using the above mentioned parameter.

Performance is always good at the previous site the developers worked at. It doesn't matter that site has long since moved to other products, after giving up in disgust with the overhead of the product and the impossible requirements of running multiple copies of production in a development system 10 times smaller - you do NOT need a big machine for development, right?

And so on.

You see: there ARE parallel universes, after all! Hawking predicted it, PSPU and SAPPU are the proof.

1:54 AM  
Blogger Moans Nogood said...

Hey Noons,

Sounds familiar :-). PeopleSoft is of course now owned by Oracle, and part of the interesting discussions going on internally are about whether to have rules, constraints, even code, in the kernel or not.

From the beginning PS wanted to be totally database-ignorant while Oracle's EBS wanted to use it to the max. So take code out of EBS or put code into PS? Big decision.

I've offered Oracle to help them with their SQL Server-based PeopleSoft-installations, but they don't seem interested :).

Mogens

6:58 AM  
Anonymous Kevin Lidh said...

Reading about the other PU's, I'd swear you were talking about Seibel. The company I worked for sent the DBAs to a class where they bragged about Fourth Normal Form, database independence, etc. The SQL I saw was very frightening and performed poorly. Also over-indexed presumably so it would be ready for any possible implementation. And you weren't allowed to remove any of the indexes even with proof they weren't being used lest you lose support. Did I discover another PU?

6:29 AM  
Anonymous Anonymous said...

Russell era again winter bootsafter recovery. Thus, although the data in NBA history, ranking Ecco Casual Shoes

4:38 PM  
Anonymous designer handbags said...

I really tried to tell Walldorf (that's where the support guys of SAP are) that it was wrong, wrong, wrong of them to keep insisting on increasing the BCHR, but they of course insisted that if we didn't follow their instructions we were not supported anymore.

5:56 PM  
Blogger allkindsofhandbags said...

2945abc45 0212

5:57 PM  
Blogger 2011bagnews said...

I have often found true religion jeans outlet the young shieldscxy cowbunting in the nests of small birds; and have seen these last followed by the young foundling, calling out clamorously for true religion sale food; and I once took a very fine one from the nest of the true religion jeans sale Maryland yellow throat, where it was fostered with great care. The migrations of these birds extend very fa? north. true religion jeans for men On their way they frequently stop in June, and are observed loitering singly, among diesel jeans thickets, reconnoitering no doubt for proper nurses to whose care G-star jeans they may True religion jeans true religion mens jeans outlet commit the hatching of their eggs, and the rearing of their helpless orphans. Among the birds selected for this duty are the diesel jeans outlet redeyed and whiteeyed flycatchers, the chipping sparrow, the goldencrowned thrush, the bluebird, the small blue gray flycatcher and and the yellow throat. mens true religion jeans The yellow throat and the redeyed flycatcher, appear to be particular favorites; and the kindness and affectionate attention which those two little birds pay to their nurslings, fully justifies the partiality of the parents. What reason nature cheap jeans may have for this extraordinary deviation true religion kids from her general practice, is beyond my comprehension.These birds often frequent corn and ricefields; but are more commonly found accompanying the skinny outlet cattle, feeding on the true religion jeans cheap seeds and worms, &;c., which they pick up amongst the fodder, &;c. Hence they are called cowbirds, •cowpen birds, and crow blackbirds. G-star jeans outlet They are generally found associated with the redwinged blackbirds, which they in many respects resemble.In the month jeans outlet of July, says Wilson, I took from the nest of a Maryland yellow throat, a young male cowbunting, which filled and occupied the whole nest. I took the bird home with me, and placed it in the same cage with a redbird, who at first and for several minutes after examined it closely and seemingly with great curiosity. It soon became clamorous for food, and from that moment the redbird seemed to adopt it as his own, feeding it true religion outlet with all the assiduity and tenderness of the most affectionate nurse.

11:59 PM  
Anonymous Best pills to delay ejaculation said...

Seriously speaking dude, I don't know what were you actually talking about out here.

3:19 PM  
Anonymous Male breast reduction pills said...

Performance is always good at the previous site the developers worked at. It doesn't matter that site has long since moved to other products, after giving up in disgust with the overhead of the product and the impossible requirements of running multiple copies of production in a development system 10 times smaller - you do NOT need a big machine for development, right?

10:21 AM  
Anonymous sivilce nasıl geçer said...

this will help me ver musch thanks a lot

7:21 AM  
Anonymous Vimax penis enlargement pills said...

Every thing seems really interesting. I appreciate your effort. Great job done.

9:21 PM  
Anonymous Male Extra enhancement pills said...

To compensate for the fact that it's old and silly technology, it's also exceedingly expensive.

1:57 AM  
Anonymous VigRX penis size enhancement gel said...

It appears what i suspected is true, all I need to be a SAP admin is pretend its 1992 and check that buffer cache hit ratio.

10:46 PM  
Anonymous ambersbridal said...

Nice article, thanks for sharing this information. Good to know that this blog is being covered also in this web site.Who wrote the sonnet of seaside romance? Guess Shakespeare would feel wordless at the sight of marine motif wedding dresses 2012. Amber’s beach cheap wedding dresses plus size unfold pictures of sea-inspired details like pearl beading, shelly appliqué and metallic embroidery. A profusion of drapes, bubbles, pleating and ruffling echoes to the very physique of the beach and the sea.

7:55 PM  
Anonymous Provillus stop scalp hair loss fast said...

They estimate that of the 1.4 million iPhones sold so far, about 250,000 of them are bought for the purpose of being resold.

4:12 AM  
Anonymous Buy Maxocum sperm pills said...

Owww. I just read a amazing article .Thanks for share it with us .

7:46 AM  
Anonymous Best Charcoal Smokers Review said...

Thank you for sharing this information with us and i hope that your future articles will impress me as much as this one.

10:08 AM  
Blogger Unknown said...

But I may be one who does not careEver to have tree bloom or bear.
Leaves and bark may be tree enough.Some giant trees hidh81ke have bloom so small.
Wholesale New Era Hats
Cheap 59fifty Hats
Cheap New Era Hats
New Era Snapback Hats
New Era Fitted Hats

6:57 PM  
Anonymous price per head online said...

Good article, read with great interest.

7:36 PM  
Blogger Unknown said...

ugg boots
true religion outlet
louis vuitton outlet
golden state warriors jerseys
adidas yeezy
louis vuitton paris
michael kors
ralph lauren polo
seattle seahawks jerseys
red bottom shoes
201612.9wengdongdong

7:57 PM  
Anonymous Obat Pelancar Haid paling Ampuh said...

It is great to have visited your website. Thanks for sharing useful information. And also visit my website about health. God willing it will be useful too

Cara Mengobati Cacingan pada Anak
Cara Mengobati Usus Buntu
Obat Benjolan di Belakang Telinga
Obat Telinga Berkerak dan Berair
Cara Membersihkan flek di paru-paru
Obat Koreng Bernanah Alami
Obat Sakit perut melilit paling Ampuh

6:40 AM  
Blogger te12 said...

qzz0727
giuseppe zanotti
christian louboutin shoes
supreme shirt
nike shoes for women
fitflops sale clearance
cheap jordan shoes
coach outlet online
dsquared
ray ban eyeglasses
ralph lauren outlet

2:02 AM  
Blogger Rohan said...

If you are facing any issue for yobit. Call us:Yobit Customer Support Number 1-(866)-828-0073 USA for getting the best customer service online and resolve all yobit account issues.

Yobit Support Number
Yobit Support Phone Number
Yobit Helpline Phone Number
Yobit Phone Number

4:51 AM  
Blogger Unknown said...

Binance issues whether complex or easy interrupts the work of users. Users sometimes need proper assistance to cope up with email verification not arriving in Binance account. In such scenarios, users can go for Binance Support and get connected with nimble and proficient geeks who not only give apt guidance but also put all their hard work to get accessible solutions for the users in minimal time. They ensure that users are enjoying their services

1:22 AM  
Blogger yanmaneee said...

yeezy boost 350 v2
curry 5
adidas stan smith
adidas superstar
retro jordans
balenciaga shoes
harden shoes
coach outlet sale
yeezy shoes
nike shoes

9:38 PM  
Blogger John smith said...

Microsoft Office is a popular office.com/setup productivity software suite. It provides various apps that include Word, Excel, Outlook, PowerPoint, and many others. The main reason for the popularity of MS Office is that its apps are very adaptable and flexible. Its desktop versions come for PC and laptops that run Windows, macOS, and Linux. Office 365 can be accessed from a web browser. Certain Office apps can also be downloaded on iPhones and Android phones. To download, install, and activate the Office product, simply click on the link office.com/setup.John Smith

12:42 PM  
Blogger Markjames said...


At Marc Sangeet Guru you get the Best Dance Choreographer who are highly talented and well trained.

Famous Choreographer
Jaimala Theme

1:34 AM  
Blogger Best Wedding Choreographer in Delhi (Dance Team By Vickky) said...

thanks shearing your information through this blog relay your information is very used full all dance choreographer. I am same services provider let take more information please visit my website

WEDDING CHOREOGRAPHER IN DELHI

4:57 AM  
Anonymous Anonymous said...

This is a wonderful post. As a writer, I work very hard to produce great articles. Inspire by this post, I'll soon write some more. There was a post on a profile I liked titled fake snapchat Generator . Snapchat's generator allows you to customize your posts in various ways. You will need to send your snaps, images, chats, and avatar. The creation and manipulation of a fake Snapchat account. In case you are interested in the same type of information, check out this profile.

2:39 AM  
Blogger Jessica Lucas said...

Expert Market Research Shared new report the name is, ‘Global Aluminum Composite Panels Market Report and Forecast 2022-2027’, gives an in-depth analysis of the global Aluminum Composite Panels Market, assessing the market based on its segments like top coatings, applications, and end-user industries, and major regions. The report about the latest trends and upcoming strategy in the industry and studies their impact on the overall market. The thriving building and construction industry owing to the increased investments in residential and commercial construction projects is driving the market growth of aluminum composite panels.

4:18 AM  

Post a Comment

<< Home