Risk Without Reward

Agreed Upon Procedures

A number of our recent posts have focused on the challenges of the hedge fund administrator's role in relation to security valuation.  We will, of course, return to this topic - but, in the meantime, wanted to focus on some of the alternatives to administrator pricing.

One of the more common comments from today's administrators is that, while an admin may be able to price Level I and Level II securities, they do not necessarily have information to price Level III instruments.  (To recap, the US accounting standard FAS 157 divides portfolios into three levels, being Level I, liquid instruments readily priced from a pricing feed (typically exchange traded); Level II, instruments priced using inputs from "comparable" securities (essentially mark to model, albeit with mainstream models); and Level III, everything else).

This leaves investors with a challenge - if administrators cannot price Level III instruments, who can? Moreover, to repeat one of our frequent comments, it is self evident that if a hedge fund manager wishes to deliberately mismark securities, they would most likely misprice a Level III instrument.  It is, of course, very hard to fake the price of IBM common stock, but much easier to mismark emerging market private loans.

Two of the most common tools available to hedge fund managers looking for third party oversight over pricing for Level III instruments - assuming the administrator has washed their hands of the problem - are third party pricing agents and auditor agreed upon procedures, or "AUP".  We will return to the strengths and weaknesses of third party pricing agents in a subsequent post, but wanted to focus this discussion on AUP.

In an Agreed Upon Procedures engagement, the auditor completes specific procedures which have been dictated by the client.  The procedures are specified and the auditor then prepares a report outlining the findings of that specific work.

We have two comments here: the first is to take a high level view as to the adequacy of these procedures, and the second is to dig a little more deeply into the actual audit guidance that covers this type of work. 

Our first comment is, unfortunately, an Emporer Has No Clothes observation.  The significant majority of hedge fund AUP engagements we have seen require the auditor to test a fund's pricing on a quarterly basis.  This usually involves (i) obtaining a portfolio list from the investment manager and (ii) testing the pricing support for those positions.

There are, however, generally two snags.  Firstly, many AUP only test a sample of prices, not the whole portfolio.  Sample testing clearly provides much less assurance than a price review of all positions: the administrator, for example, is usually expected to price the entire book (would any investor accept a NAV which has been priced on a "sample" basis???)

The bigger problem, however, is the type of testing completed by the auditor.  In way, way too many cases, the auditor tests security prices back to the manager's own pricing support and makes no attempt to obtain independent pricing information.

This type of work is, clearly, somewhere between minimal and absolutely no value for investors.  If the auditor receives a spreadsheet from the manager showing the matrix of broker quotes received, how does the auditor know that the manager has not adjusted that spreadsheet to exclude quotes which were uncomfortably low?  Even more importantly, if all the auditor does is to check prices back to pieces of paper in the manager's own pricing file, how does the auditor know that those pieces of paper are genuine?  As we have said before, and will keep on saying, it only costs $500 to buy a copy of Adobe Photoshop if you are of a mind to alter documentation.

When discussing this type of work, the manager typically notes that, if the auditor was to complete a full, independent pricing review, it would be too costly and too time consuming to be practical on a quarterly basis.  A full, GAAP audit review is, of course, performed at year end - this does include independent pricing (although - investor fyi - auditors will still only sample test many portfolios.)

While these are fair points, it remains the case that this type of AUP provides minimal protection against pricing fraud.  In the meantime, the manager gets the marketing benefit of being able to claim enhanced scrutiny and oversight from a Big 4 firm each quarter.

Which leads to our second point.  Why would an auditor accept to complete agreed upon procedures when any reasonable accountant would rapidly conclude that the typical scope of these AUP provide pretty much nil controls assurance?  Why does the auditor not insist that, if their name is to be associated to this work, then the procedures must be meaningful and sufficient to meet an actual control standard?  

To this point, the actual audit standard applicable to AUP is available here.  The standard states:

An agreed-upon procedures engagement is one in which a practitioner is engaged by a client to issue a report of findings based on specific procedures performed on subject matter. The client engages the practitioner to assist specified parties in evaluating subject matter or an assertion as a result of a need or needs of the specified parties. Because the specified parties require that findings be independently derived, the services of a practitioner are obtained to perform procedures and report his or her findings. The specified parties and the practitioner agree upon the procedures to be performed by the practitioner that the specified parties believe are appropriate. Because the needs of the specified parties may vary widely, the nature, timing, and extent of the agreed upon procedures may vary as well; consequently, the specified parties assume responsibility for the sufficiency of the procedures since they best understand their own needs. In an engagement performed under this section, the practitioner does not perform an examination or a review, as discussed in section 101, and does not provide an opinion or negative assurance. Instead, the practitioner's report on agreed-upon procedures should be in the form of procedures and findings.

In practice, this all gets horribly circular.  Per the standard, a client requests an auditor to complete AUP to assist "specified parties" to "evaluate subject matter or an assertion".  In our case, the assertion would be "are hard to value securities valued correctly at quarter end."

However, the specified party is usually the manager itself, making the client and specified party the same person.  The particular trick applied, in many cases, is for the auditor to seek to prevent the investor from actually seeing the AUP in the first place!  However, if the investor is to have access to the AUP, the auditor universally requires the investor to sign a Catch 22 document which requires the investor to acknowledge that the AUP are "sufficient for their needs".  So, even if the investor believes that the AUP are not "sufficient for their needs" - which is hardly a long stretch - the investor has to sign that the procedures are sufficient if they are to even see the auditor's work.  With this magic piece of paper, the auditor has met its requirements and can sleep easy.  Meanwhile, the auditor will send a bill to - guess who - the fund, meaning that investors have, once more, had to foot the bill.

As always, Caveat Emptor.

www.castlehallalternatives.com

Hedge Fund Operational Due Diligence

And the auditors work for....

Audit opinions of a hedge fund's financial statements are unlikely to make the New York Times bestseller list.  As a result, we can certainly understand if the auditor's fine print is not exactly top of the list for investor attention.  However, not all audit opinions are the same and, over time, it seems that different audit firms are quietly introducing different standards of care and attention - and, of course, liability, which is always the 800 pound gorilla.

The issue is the addressee of the audit report - or, put more simply, who the auditor works for.  In a public company, the auditors report to both the shareholders and the Board of Directors.  A quick web search gives us a couple of examples - GE and Goldman Sachs (don't laugh at the Level III assets, by the way).

In a hedge fund, however, sometimes the audit report mentions the shareholders, but sometimes it does not.  What seems to be a fine difference is actually very profound - exactly why would a hedge fund auditor report only to the Board of Directors and deliberately fail to address their report to the shareholders?  Adding insult to injury, of course, is the reality that the average Board of Caymanian rent-a-directors hardly acts with the same vigor and intervention as the non execs on the boards of GE and Goldman.

In our experience, certain audit firms appear to have taken a deliberate decision to direct their audit opinions, wherever possible, only to the directors.  This is a difference which applies across both US GAAP reports as well as audits completed under International Financial Reporting Standards.  Check 10 audit reports from different firms, and see what we mean.  

The underlying issue - of course - is the lack of investor control.  Investors, if asked, would very likely have an opinion on this issue: but, needless to say, they are not asked.  Audit engagement letters are signed under cloak and dagger secrecy (usually because they include ever more expansive terms seeking to limit auditor liability under Caymanian law).  Thereafter, as investors and due diligence practitioners know to their ongoing annoyance, it proves incredibly difficult and pointlessly time consuming to get some auditors even to confirm that they are the auditor of record for the hedge fund in question.

In the short term, one answer would be for offshore jurisdictions such as the Cayman Islands to mandate that all audit reports filed for Caymanian hedge funds be addressed to the shareholders rather than just the Board.  If it's good enough for GE, it should be good enough for any hedge fund.

In the bigger picture, however, this is just one question within the broad construct of Hedge Funds 2.0 post Madoff.  Unfortunately, it is only investor pressure which can enforce any change so that service providers - auditors, administrators, lawyers et al - take responsibility and recognize that their primary duty of care is to the investors that pay them.  Without that pressure, hedge funds will continue to be the asset class where everyone wants to get paid, but no-one wants to take responsibility.

www.castlehallalternatives.com

Hedge Fund Operational Due Diligence

OK, Mr. Auditor....

The Wall Street Journal published an excellent summary of the challenges faced by hedge funds in 2008 a few days ago (available here.)

The key trends are familiar: poor performance; fund closures; "herd mentality" as too many funds chased the same investment ideas; Madoff; and funds which prevented investors from redeeming their capital.

The article also comments on issues which will impact investors going forward.  Notably, Espen Roback, President of the specialist (and in our humble opinion, extremely good) independent valuation firm Pluris Valuation Advisors is quoted talking about hard to value securities and the role of the auditor.

"One looming problem for many hedge funds is the amount they still hold of hard-to-trade assets, such as loans, real-estate holdings and stakes in small, private companies. These illiquid investments at one time accounted for 20% of some fund portfolios, estimated to total about $400 billion. As financial markets have come under pressure, it has become much harder to get out of these investments, or even to value them accurately. That could hurt hedge funds as they try to attract new investors in 2009.

"Auditors do not, contrary to popular assumption, control a fund's valuation policy; management does," said Espen Robak. "An auditor auditing five different funds may sign off on five different valuations for a single investment."

We have often made exactly the same point as Espen.  In our post on FAS 157 on November 8, 2008, we noted that:

"On this point, investors should also remember that the auditors of underlying hedge funds are not responsible for those funds' accounting policies: those, too, are set by the management of the underlying fund.  As such, different funds holding the same security, audited by the same audit firm, may well hold that same security at different prices, yet still receive unqualified audit opinions.  The auditors are not, to be clear, imposing any form of pricing standardization or consistency: the auditors do not have a central master pricing file that they force all hedge funds to use."

As an ex auditors ourselves, we are very familiar with all the challenges involved in auditing hard to value securities.  FAS 157, in turn, introduces many new, extremely difficult issues.  Nonetheless, this lack of consistency across audited financial statements, in our view, needs to be far more widely emphasized.

There are two problems.  Firstly, hedge fund managers are very ready to tell investors that the auditors "signed off on our valuations" and that any investor concern over valuation is unjustified as the "auditors were happy".  Audit firms cannot live in a bubble - they have to remember that the audit is used as a key bargaining chip in the push and pull between manager and investor during the due diligence process, as investors try to get comfortable over the pricing of often hard to value, opaque portfolios.

The second issue is more systematic.  Investors should always remember that the auditor is hired and fired by the manager, not the investors, and throughout the audit process has contact only with the manager and no contact whatsoever with investors.  As such, it is the manager who is the "client", and the party who has to be kept happy.

This actually reflects the core principle of auditing, being that the financial statements are the responsibility of management, not the auditor (and certainly not the investor.)  All the auditor is hired to do is to form an opinion as to whether management's accounts are prepared fairly in accordance with GAAP, within a boundary of materiality (which, by the way, is never disclosed to investors.)  The auditor is emphatically NOT hired to prepare the accounts themselves in their own preferred format, with the best numbers they can come up with independently.

Thinking about the audit process simplistically, it is possible to approach an audit in one of two ways.  On the one hand, the auditor can seek to gather sufficient evidence to justify the manager's accounting: can we find evidence, for example, to support the manager's stated security valuations.  On the other hand, the auditor can seek out audit evidence which actively tries to disprove the manager's accounting and financial statement presentation.  Clearly, these are two very different things.

In reality, this is not a binary choice and most audits operate in a continuum somewhere between these two extremes.  The expectations gap, however, is that most investors assume that the focus of the audit is always and solely on the latter point, being a concerted effort to disprove the manager's accounting.  In reality, this is not the case at all.  

Against this varied background, the roadblock for investors is that it is impossible to evaluate the audit in any way.  If we think hedge fund managers are opaque, they actually operate in the full "sunlight of public scrutiny" compared to the audit firms, who provide absolutely no transparency into their work at all.  Investors cannot speak to the audit firm to understand the scope of the audit (which may vary widely between firms and funds - did, for example, the auditor reprice all portfolio securities, or just reprice a sample?).  They cannot evaluate the quality of the audit team to understand whether the partners, managers and seniors actually know anything about hedge funds.  They cannot review the audit working papers.  They cannot review the summary of unadjusted items (the changes the auditor had proposed but which were passed on the grounds of materiality.)  As above, they cannot evaluate what materiality actually was.  They cannot discuss the audit with the partner to determine whether this was a well run and clean audit, or whether the hedge fund manager was a nightmare who got their audited accounts only by the very tips of their fingernails.

In fairness to the auditors, one of their problems is the binary nature of the audit opinion itself - you either provide a clean audit opinion or you don't (we'll ignore the nuances of emphasis of matter paragraphs which don't really impact the overall "pass / fail" framework in any case.)  There is no reporting framework which allows the auditor to express their professional opinion on the more nuanced aspects of the audit - how good actually is the manager in question?  Was this a 92% or a 27% audit in terms of overall quality and confidence?

In reality though, the auditors would never want the liability to provide these opinions anyway, so the whole thing is a moot point.  Our message to investors, therefore, is to treat the audited financial statements as a tool in the toolbox.  They are important, but they are not, in any way, absolute - just because a hedge fund has a clean audit, it does not mean that that hedge fund has followed best practice accounting, operational or reporting practices.  We would also add that the auditors' self appointed status as the omnipotent "gurus" of all accounting and operational issues - and the Big 4 are terrible for this - should be taken with a grain of salt.

In fact, what really started us thinking about the whole audit process was an innocent question from one of our clients a few days ago.  In disbelief over Madoff and the tragic audit failures in that debacle, they asked us if we had ever seen a set of hedge fund financial statements which had received a qualified audit opinion.

We all know the answer to that.

www.castlehallalternatives.com

Hedge fund operational due diligence.  

And some other resources and publications links...

Having just touched on both EY and Grant Thornton, we thought we'd be fair to the other accounting firms and also give ourselves a quick plug. Castle Hall's website includes a resources page with links to various industry publications, including papers from the accounting firms and other sources related to valuation issues and AU 332. Feel free to e-mail us if you have any suggestions as to other publications which could be added to the list.

Grant Thornton Hedge Fund Adviser

We thought we would highlight a quarterly newsletter produced by the "Big 5" audit firm Grant Thornton (OK, accountant's joke, sorry!) "Hedge Fund Adviser" (available here) is published quarterly and has touched on some very useful topics, including FAS 157, the Cayman electronic reporting regime and most recently the new IRS rules related to deferred compensation. For those interested in such things, this is a very helpful resource.

Ernst & Young hedge fund survey

Ernst & Young recently published their 2007 “Global Hedge Fund Survey” (available here). This is a survey of approximately 100 hedge fund managers and outlines their views on some of the current and future challenges facing the industry. It’s something of a scattergun, covering everything from distribution channels, fee structures and the need for permanent capital through to the degree to which managers are happy with their fund administrators.

EY uses the term “industrialisation” to refer to the changing industry, apparently in an effort to be clever and avoid the oft repeated mantra of “intitutionalisation”. What EY really means is that hedge funds are morphing into asset management businesses - with the procedures, systems and infrastructure that suggests - rather than continuing as looser, entreprenuerial entities designed primarily to give free expression to the portfolio manager’s intellect.

EY’s findings give great emphasis to business and operational issues. According to the survey, the single biggest issue for managers is managing operational risk, followed by attracting and retaining talent (after all, you never find the untalented working at a hedge fund!) Managing the growth of the firm and the regulatory environment also weigh as greater concerns than the fifth factor, attracting new investors.

At the same time, the largest group of managers surveyed - some 37% - believe that they will need to make no change at all to the operational side of their business to accommodate changes in the profile of the firms’ investor base (i.e. institutions and other more "demanding" investors). The next biggest group (only 15% of respondents) believe they will need to create a “more formalised control structure”.

EY notes that this “confidence is surprising since it seems to conflict with responses elsewhere on infrastructure, and our experience of benchmarking processes and procedures to those demanded by best practice or regulation.” In other words, perhaps there is a trend to talk the talk but not walk the walk.

The main sources of operational risk are seen by managers as:

Not having the right people (18%)
Valuation (16%)
Complexity of new products and strategies (14%)
Compliance with regulatory changes (14%)
Inadequate technology (13%)

We are astonished that only 16% of the managers surveyed see valuation as a source of operational risk over the net two years. To us, pricing is the central operational issue facing the industry and far, far more than 16% of hedge funds hold hard to value securities. Many managers will face internal challenges when pricing their portfolios and, over the next two years, firms will also have to respond to more demanding expectations from investors looking for far better control over this critical area.

When managers are asked to identify the steps they will be taking to improve the management of operational risks, the list becomes:

Hire more of the right people (34%)
Invest in technology (24%)
Improve or institutionalise policies and procedures (15%)
Bring in outside expertise (11%)
Review control processes (11%)

6% see no steps necessary.

We have one final comment - while surveys of this type are always helpful (and investors should be grateful for the time and energy the sponsors put in), they are consistently manager focused. More broadly, the same goes for many of the industry "best practice" initiatives - AIMA, the MFA and the UK Hedge Fund Working Group all represent the manager rather than investor communities.

We would be very interested to see a survey which polls investors as to their expectations. While we know what managers want to do, could we have some more information about what investors would like them to do?

Understanding the new audit standards

And finally....for those who have interest in such things, Rothstein Kass have published a presentation which discusses new US audit standards and their impact on hedge funds.

Among the standards is AU 112, "communicating internal control related matters."  This requires that any "significant deficiencies" or "material weaknesses" identified during the audit process must be communicated to management and "those charged with governance".   For investors, therefore, a new question is to request access to the AU 112 letter, or to receive confirmation that no such letter was issued.  Of course, if the manager doesn't want to let the investors view the document.....