As a CFO and senior business executive would life be better if all your commercial information, everything from files and documents to LOB systems, were all accountable and controllable from one platform or portal?

Imagine definitive decision support, comprehensive implementable governance, anything and everything at your finger tips via a dashboard and ‘best bet’ query.

In recent times, and with its ubiquitous commercial acceptance showing no sign of abating, SharePoint has been moving to take this kind of ‘center stage’ in the information management universe.

But in many cases SharePoint is filling this role by default, due to its momentum within IT and other business areas, without a comprehensive assessment as to its suitability for this pivotal role.

Could SharePoint be a black hole for your business, drawing in information and content without providing an adequate foundation for information management?

Let’s look at the typical benefits SharePoint delivers then consider some of the strategic implications these benefits portend.

Typical SharePoint Benefits
There is no doubt that SharePoint delivers extensive business benefit in the information worker and general office productivity area:

• Document Libraries – Business content such as documents, spreadsheets, graphics, presentations, media files, even emails and their attachments, can be housed and ‘managed’ in SharePoint libraries easily accessible by anyone, anywhere, anytime, via intranet, Extranet or Internet

• Roles & Controls – Library controls, and permissions across SharePoint, enforce author, editor, publisher roles and structured responsibilities along with fail-safe versioning, garbage, archival repository and other content controls

• Search & Surfacing – Search promises that nothing will ever be ‘lost’ again. Anything in your Enterprise in SharePoint, and beyond, can be found or ‘surfaced’

• Security – Security is leveraged from your existing Active Directory investment or other authentication system integrating seamlessly with content roles and controls. Single sign-on can make any LOB system, such as financials, ERP or CRM, directly accessible or integrated with your portal

• Information Management – Metadata can be tailored to your business and assigned to assist with search, categorization, and information processing

• Customisation -SharePoint is extensible with features and functionality that can be exposed and customized, or created for example as Web Parts and seamlessly integrated

Implications
But what are the implications behind these SharePoint benefits and how do they impact your business?

Let’s consider three basic SharePoint issues that span the business technology ‘sophistication’ spectrum from: the basics of content library storage; to custom development; to business intelligence

SharePoint’s Storage Paradigm Shift
While the concept of computer storage might be considered somewhat boring and ‘something for the Techs’, it is important to note that SharePoint involves a massive paradigm shift in this area.

In the past, Fileshares have been the most common method of general storage. But their time has passed and it is now almost universally considered a good idea to have all your content in record management systems, document libraries and workspaces.

But where does all that content actually live once it is in SharePoint in a document library?

Fileshares in Windows, and folders in SharePoint, while functionally similar are very different entities from a storage perspective

Doesn’t SharePoint store content in its own folders and ‘fileshares’, the way it appears on screen?

No. SharePoint stores all its content in a Microsoft SQL Server database.

SQL Server is not a new technology and your business likely already has SQL Server running, but consider the implications of this change:

• Licensing Cost – SQL Server involves its own licensing costs both for the purchase of the software and CALS (client access licenses). This is in addition to the SharePoint costs. By contrast storing and sharing files in folders is essentially ‘free’

• Infrastructure & Performance – SQL Server is a sophisticated platform that necessitates specialised infrastructure including servers, storage devices, and staff, in addition to SharePoint. The cost for the use of this infrastructure may be spread over other applications that are already using SQL, but if your business is looking at putting all its content into SharePoint+SQL then this infrastructure will have to scale both in terms of its size and mission critical support.

• If you require the SharePoint content to be available on an operational basis special attention will need to be given to SQL I/O performance along with addressing problematic performance issues with SharePoint itself, such as the performance penalty when returning more than 2,000 records in a list. Performance limitations might make certain types of enterprise-wide solutions untenable. Capacity planning is essential. Here is a useful link on SharePoint capacity planning.

• Migration – While there are tools that allow the migration of content into SharePoint from earlier versions of SharePoint and other information sources such as Lotus Notes, this is typically not an easy or ‘lossless’ exercise. Looking to the future, will there be tools to migrate your content out of SharePoint+SQL if needed? What about all the feature rich content attributes you might build into your SharePoint information management solution – will you be able to take them with you?

Custom Development Dilemma
SharePoint is a great OOTB, ‘out of the box’, productivity tool. But OOTB functionality is never enough! Customization is always demanded to support specific business processes and applications.

However, while SharePoint has a wealth of inbuilt functionality waiting to be tapped, commensurate with the richness of features there is a high degree of complexity that is largely underrated. Development in SharePoint brings its own challenges:

• Business Coverage- SharePoint covers many different business areas including content management, business intelligence, document and records management, workflow, portals, etc. Ideal SharePoint solutions should leverage as much of the existing platform functionality as possible without reinventing the wheel.

• For best results a developer should have a holistic functional understanding of SharePoint across all its business applications

• Complex Technology – SharePoint is a complex technology in its own right that spans from web front ends to SQL Server database processes. The right approach, expertise and understanding are required to fully leverage it. Good .Net developers are not automatically good SharePoint developers. Experience, lots of experience, is needed to bring about holistic development understanding. Experience also assists in coping with SharePoint’s many ‘undocumented features’.

• Because many SharePoint options can be configured without development, and initial development appears easy, overconfidence is often engendered in in-house developers, and overly ambitious development projects can be undertaken with disastrous results

• Duplicitous Paths -With Microsoft technologies there are often several development paths to achieve a solution and SharePoint is no exception. But despite the seeming logical and extensible nature of a path, a satisfactory result may not be achieved. As this stage of SharePoint’s maturity, many likely development paths often come to a dead-end through no fault of the logic of the developer or their approach, but because of a SharePoint bug, anomaly or undocumented feature.

• SharePoint is only early into SPK release lifecycle and the many CUs (cumulative updates) are a must

• Development Environment – a suitable development environment is required which ideally should replicate Production conditions which may involve topologies of web farms, index servers and SQL clusters, Active Directory and Exchange servers. Few environments provide that level of development platform support, although virtualization is closing the gap. This means that SharePoint development projects do not always travel well to the Production environment, even with Solutions and Packages, delivering unforeseen results

SharePoint Production environment topologies can be complex, and are not often replicated in development, with the result that SharePoint solutions do not always travel well when deployed

Metadata Intelligence
The leveraging of metadata is one of the most underrated areas in SharePoint yet it is invaluable to your business and information management strategy.

For example the implementation of an information management system, incorporating governance and compliance built on SharePoint’s metadata and workflows functionalality, is a logical application for the platform.

Metadata support allows you to attach keywords that can be used to provide meaning about the content that is loaded, assisting in categorization, search and aggregation. It is also a key to ensuring the alignment of content and business processes to your governance, compliance and information management plans, by being linked back to your associated business goals and objectives.

SharePoint OOTB can leverage the Properties page in all Office documents where metadata for author, title, subject, status, keywords, etc., can be entered. In addition SharePoint has a whole metadata infrastructure that allows you to create, customize and manage your own metadata infrastructure.

This SharePoint metadata, called content types, have particular value in that they can also be used to trigger actions and workflows. For example when a document is loaded into a library, special processes can be triggered based on its metadata.

Let’s consider some of the implications of even a basic implementation of SharePoint’s metadata on a large scale. (For ease of discussion I will use the term metadata rather than content type.)

Note the message ‘You MUST fill out any required properties’

• Mandated Metadata – It seems like a good idea to allocate metadata to all your SharePoint content because of the associated benefits. So the next obvious step is to mandate metadata i.e. ensure every piece of content that is loaded into SharePoint must be assigned relevant metadata.

• To add metadata to a single document may only take a minute but scale that out across your business to every staff member and to every piece of content, and a sizeable amount of time will be taken up assigning metadata that will erode overall business productivity

• Dirty Metadata – One way to overcome this ‘mandated metadata productivity hit’ is to make the metadata faster and more intuitive to allocate. This can be done by providing metadata in the form of dynamic lists, as opposed to free text fields.

• But even when presented with an easy or ‘smart’ pick list, staff may not know what metadata value to assign, or be confronted with too many choices. For example combinations of departments, business units, and projects could produce an overwhelming list of options.

• ‘Time poor’ information workers don’t want to stop their task to find out what metadata should be allocated or go through the process to create a new category. This can lead to erroneous values being assigned just to expedite content loading.

• In some cases there may be purposeful erroneous metadata provided, to obscure business activities such as fraud

• Legacy Metadata – Most discussions of SharePoint benefits focus on ‘greenfields’ SharePoint implementations but all businesses have legacy fileshares, content and systems that can be essential to migrate into SharePoint.

• While it is possible to programmatically assign metadata during migration, in reality it is not worthwhile to do so on any scale unless a metadata or similar policy has previously been in place.

• Consider trying to allocate even simple metadata such as author, document title and version if these are not already associated with the content: fileshare locations by nature provide multi-author storage so authors are not differentiated from PAs, etc.; duplicates and drafts with the same or entirely different titles are common confounding categorization and titling; latest date may not reflect a file master version. These are just some of the issues.

This much used image helps demonstrate the typical nature of the information and content in most businesses fileshares. Determining, on any scale, the author, title, keywords of this information is in reality an impossible task.Thus much legacy information comes into SharePoint sans the metadata that could be vital to an understanding of the material, and assist with the information management of the business

• Metadata Meaning -Metadata seeks to give meaning to content beyond that given by a file name.

• How many metadata properties might be needed to adequately indicate the meaning of a document that spans many diverse areas and topics? Free text metadata would allow more comprehensive explanation but we are back to the ‘productivity hit’ issue of the time taken to enter all the metadata.

• Meaning may also differ from different perspectives. How would the allocation of ‘definitive meaning’ to files and content be achieved? How would different perspectives and synonyms be handled where different metadata words and terms are used for similar concepts?

• Consider the issue of trying to assign comprehensive metadata to all the images, video, audio and VoIP content in your business. How much resource would be required? Would the metadata be accurate and definitive using any current methods?

• In terms of scale, how would the process of allocating metadata relating to every email be handled?

• What about metadata relationships between content, such as between documents and documents, images, emails, projects, departments, people, roles? Would this type of meaning and understanding gained provide any useful insight into your business and be of value?

These are issues that may currently seem most poignant to the legal profession and e-Discovery but are increasing in import for all businesses.

Metadata that provides meaning as described above, including that for unstructured content, can provide valuable insight into business operations providing a basis for real business intelligence, way beyond that of the traditional BI with just numbers and charts in a spreadsheet.

Conclusion
SharePoint is a good information management and general office productivity platform. However it has its issues that are best to be aware of and managed, rather than just ‘let loose’ in your business, preventing it from becoming a ‘black hole’.

In the context of the issues discussed above:

The Good is that SharePoint will, in line with the Pareto principle, OOTB meet roughly 80% of your general information management needs and in the process provide a stable scalable platform.

Good also is that assistance with SharePoint development, both in terms of available tools and developer experience, is increasing at a rapid rate. At the time of writing this article, a quick search of www.CodePlex.com returned nearly 700 ‘open source’ projects providing explanation and assistance on SharePoint development. Countless blog articles by SharePoint gurus likewise provide commentary on numerous SharePoint solutions.

The Bad is that if you want SharePoint, then SQL Server and its associated overheads are unavoidable.

Bad also is that with regard to going beyond the metadata basics, there is no solution available within SharePoint that address the issues discussed. However this is not so bad as it seems as there is no solution available within any of the current generation of similar platforms.

How to get more meaning from business information and unstructured content is the next major conundrum facing business managers today. However I believe this type of metadata support and analysis is one of the most exciting challenges, and will be the basis for the next level of real business intelligence and competitive business advantage.

Next generation tools, providing definitive meaning to structured and unstructured information and a full understanding of information relationships, are just starting to enter the mainstream and gain more attention. These tools can work with SharePoint as well as hundreds of other information sources and hold the promise of providing an incredibly comprehensive understanding and powerful information management tool for your business.

I will be looking at this next-generation solution in more detail shortly.

No business is immune from this ‘SharePoint Paradox’. Understanding and avoiding it is an essential part of SharePoint success.

SharePoint Popularity

Did you know that SharePoint 2007 is Microsoft’s most popular product ever, having achieved the $1billion 100 million sales mark faster than any other product in Microsoft history? A position achieved with none of the marketing hype that accompanies the likes of Office or Windows!

The popularity of SharePoint is built on a number of characteristics, including:

• it has a wealth of easy-to-use, web-based collaboration tools and templates that are analogous to Outlook and so receive ready uptake, while providing much more functionality
• it is easily extensible and customizable to fit your business
• only limited IT involvement is needed
• it is a real 80-20 tool, almost 80% of common business tasks can be achieved OOTB (out of the box)
• it integrates seamlessly with Active Directory, and with a bit more effort to your LOB applications, providing immediate secure use enterprise-wide
• it can scale easily

SharePoint truly is a ‘wonder tool’ of the information age.

It can takes less than 30 minutes to install SharePoint, create a website for collaboration, and start enjoying the benefits of collaborative resources such as task lists, calendars, versioned document libraries,  workflows, CMS publishing.

Where’s The Problem?

So where’s the problem?

The paradox is that SharePoint ‘s inherent ease-of-use is its own worst enemy. Rapid, organic, unstructured SharePoint growth leads to business disaster.

This pic shows the APAC SharePoint conference with eight Microsoft team members on stage including Mike Fitzmaurice, Joel Oleson and Angus Logan. SharePoint while initially appearing easy and straight forward is a sophisticated and complex technology requiring a depth of expertise

Consider the following post-installation problem scenarios based of real life experience:

• § SharePoint websites can be setup by a non-IT user in 5 minutes without requiring any pre-qualification or adherence to business mission or information architecture. Hundreds of non-business-aligned, duplicate and redundant sites tend to proliferate. The overhead in rationalizing is substantial
• § Document libraries are created by default and are easy to populate. Nomenclature and meta data standards are not automatically applied, and bad habits associated with existing file shares are often transported into the document libraries. Document search and manageability, precisely the issues meant to be improved, become bigger
• § The SharePoint install is so easy that the subsequent expertise needed to effectively configure and govern SharePoint within the business is often dramatically underestimated until it is too late
• § Once installed, the ongoing management of SharePoint can escape the rigors of IT because of its ready take-up and hand-over to the business

What To Do?

Without effective and specialized governance, SharePoint will only replicate your business’ existing problems at a faster rate and on a larger scale than you thought possible!

You need a governance plan enacted in your business by an appropriately empowered steering committee.

This SharePoint Steering Committee should have enough seniority to ensure two-way business alignment, ie that SharePoint is aligned with the existing business governance plan and associated initiatives like the information architecture; and that the business plan can be updated to incorporate benefits derived from SharePoint.

The ubiquitous and innocuous File Share. To some it may seem strange that just loading hundreds or thousands of documents from their existing file-shares into a SharePoint document library doesn’t solve the problem of not being able to find things!

This sounds pretty straight forward. Just apply a SharePoint governance plan and all will be okay.

There are literally thousands of pages of information available on SharePoint governance. The Microsoft SharePoint Governance Plan and Check Listare a good place to start. An Internet search will return tomes of material on governance with some documents in the realm of 600+ pages in length!

Why Problems?

Why are problems still encountered?

The answer to this is two-fold.

First, with this seeming wealth of governance related material available, none is definitive, and most discusses what governance should do, not how to do it!

For example, we discussed above that SharePoint should be aligned to the business governance plan and information architecture. But how do you do that?

Similarly, the Microsoft Governance Plan identifies that up to 30 roles can be used to effectively govern SharePoint, but which business has the resource to cover all those roles or can encumber existing staff with that level of extra responsibility? Which roles are essential and what actions should they cover?

Second, for most Microsoft Partners, SharePoint is just one product in their already overloaded product set.  Consider the plight of the poor Microsoft Partner with SharePoint 2007, Office 2007, Windows 2008, SQL 2008 and VS2008 all released little more than 12 months apart. Which Partner let alone business has the ability to skill-up on that range of technology? ‘Heroes’ or not!

Fewer organizations have a history with SharePoint beyond the current version, and almost none can take a ‘holistic’ view i.e. discuss a SharePoint solution in the context of the entirety of its features set.

Governance plans talk a lot about ‘What To Do’ but not ‘How To Do It’. You need an extensive history with SharePoint, all its incarnations ie portal, DMS, CMS, BI, horizontal and vertical platform application, experience in its application to various business issues, strengths and weaknesses, to be able to best plan its use in the organization.SharePoint Practice Governance, based on an extensive experience of this kind of ‘best practice’ – a definitive body of knowledge- allows for the proactive rather than reactive planning of the use of SharePoint in your organization. If you don’t use deeply experienced SharePoint resources with access to this Practice Governance you are faced with trying to ‘learn on the job’ and with SharePoint’s tendency towards rapid organic growth, if you don’t get it right from the start problems will overwhelm the resources available to manage SharePoint..

It is only from experiencing SharePoint’s strengths and weaknesses across its diverse range of functionality, from portal, to BI, to DMS, to CMS, forms, from generic horizontal platform to adapted vertical solution, and how it performs across diverse business applications, that you can begin to understand and identify SharePoint best practice and effective governance practices.

SharePoint Practice = Best Practice

In short, you need a lot of ‘SharePoint practice’ to be able to provide ‘SharePoint best practice’.

Most businesses can not afford to implement systems, let alone a widespread platform like SharePoint, on a trial and error basis trying to work out the best way to do things ‘on the job’ (though some Partners often try to get away with having their staff learn that way!)

So in conclusion as you work through your SharePoint implementation and ongoing management, if it isn’t producing the business advantage and results you expect, find a partner who has a SharePoint specialistion.

Draw on their rich best practice to source and develop your governance plan, stop the SharePoint paradox, and get the SharePoint results and business success you want.

In following articles I will explore Practice Governance and the best practice it is based upon in more detail.

Till next time – Julian