SharePoint -Black Hole or Star of Your Business Universe?

As a CFO and senior business executive would life be better if all your commercial information, everything from files and documents to LOB systems, were all accountable and controllable from one platform or portal?

Imagine definitive decision support, comprehensive implementable governance, anything and everything at your finger tips via a dashboard and ‘best bet’ query.

In recent times, and with its ubiquitous commercial acceptance showing no sign of abating, SharePoint has been moving to take this kind of ‘center stage’ in the information management universe.

But in many cases SharePoint is filling this role by default, due to its momentum within IT and other business areas, without a comprehensive assessment as to its suitability for this pivotal role.

Could SharePoint be a black hole for your business, drawing in information and content without providing an adequate foundation for information management?

Let’s look at the typical benefits SharePoint delivers then consider some of the strategic implications these benefits portend.

Typical SharePoint Benefits
There is no doubt that SharePoint delivers extensive business benefit in the information worker and general office productivity area:

• Document Libraries – Business content such as documents, spreadsheets, graphics, presentations, media files, even emails and their attachments, can be housed and ‘managed’ in SharePoint libraries easily accessible by anyone, anywhere, anytime, via intranet, Extranet or Internet

• Roles & Controls – Library controls, and permissions across SharePoint, enforce author, editor, publisher roles and structured responsibilities along with fail-safe versioning, garbage, archival repository and other content controls

• Search & Surfacing – Search promises that nothing will ever be ‘lost’ again. Anything in your Enterprise in SharePoint, and beyond, can be found or ’surfaced’

• Security – Security is leveraged from your existing Active Directory investment or other authentication system integrating seamlessly with content roles and controls. Single sign-on can make any LOB system, such as financials, ERP or CRM, directly accessible or integrated with your portal

• Information Management – Metadata can be tailored to your business and assigned to assist with search, categorization, and information processing

• Customisation -SharePoint is extensible with features and functionality that can be exposed and customized, or created for example as Web Parts and seamlessly integrated

Implications
But what are the implications behind these SharePoint benefits and how do they impact your business?

Let’s consider three basic SharePoint issues that span the business technology ’sophistication’ spectrum from: the basics of content library storage; to custom development; to business intelligence

SharePoint’s Storage Paradigm Shift
While the concept of computer storage might be considered somewhat boring and ’something for the Techs’, it is important to note that SharePoint involves a massive paradigm shift in this area.

In the past, Fileshares have been the most common method of general storage. But their time has passed and it is now almost universally considered a good idea to have all your content in record management systems, document libraries and workspaces.

But where does all that content actually live once it is in SharePoint in a document library?

Fileshares in Windows, and folders in SharePoint, while functionally similar are very different entities from a storage perspective

Doesn’t SharePoint store content in its own folders and ‘fileshares’, the way it appears on screen?

No. SharePoint stores all its content in a Microsoft SQL Server database.

SQL Server is not a new technology and your business likely already has SQL Server running, but consider the implications of this change:

• Licensing Cost – SQL Server involves its own licensing costs both for the purchase of the software and CALS (client access licenses). This is in addition to the SharePoint costs. By contrast storing and sharing files in folders is essentially ‘free’

• Infrastructure & Performance – SQL Server is a sophisticated platform that necessitates specialised infrastructure including servers, storage devices, and staff, in addition to SharePoint. The cost for the use of this infrastructure may be spread over other applications that are already using SQL, but if your business is looking at putting all its content into SharePoint+SQL then this infrastructure will have to scale both in terms of its size and mission critical support.

• If you require the SharePoint content to be available on an operational basis special attention will need to be given to SQL I/O performance along with addressing problematic performance issues with SharePoint itself, such as the performance penalty when returning more than 2,000 records in a list. Performance limitations might make certain types of enterprise-wide solutions untenable. Capacity planning is essential. Here is a useful link on SharePoint capacity planning.

• Migration – While there are tools that allow the migration of content into SharePoint from earlier versions of SharePoint and other information sources such as Lotus Notes, this is typically not an easy or ‘lossless’ exercise. Looking to the future, will there be tools to migrate your content out of SharePoint+SQL if needed? What about all the feature rich content attributes you might build into your SharePoint information management solution – will you be able to take them with you?

Custom Development Dilemma
SharePoint is a great OOTB, ‘out of the box’, productivity tool. But OOTB functionality is never enough! Customization is always demanded to support specific business processes and applications.

However, while SharePoint has a wealth of inbuilt functionality waiting to be tapped, commensurate with the richness of features there is a high degree of complexity that is largely underrated. Development in SharePoint brings its own challenges:

• Business Coverage- SharePoint covers many different business areas including content management, business intelligence, document and records management, workflow, portals, etc. Ideal SharePoint solutions should leverage as much of the existing platform functionality as possible without reinventing the wheel.

• For best results a developer should have a holistic functional understanding of SharePoint across all its business applications

• Complex Technology – SharePoint is a complex technology in its own right that spans from web front ends to SQL Server database processes. The right approach, expertise and understanding are required to fully leverage it. Good .Net developers are not automatically good SharePoint developers. Experience, lots of experience, is needed to bring about holistic development understanding. Experience also assists in coping with SharePoint’s many ‘undocumented features’.

• Because many SharePoint options can be configured without development, and initial development appears easy, overconfidence is often engendered in in-house developers, and overly ambitious development projects can be undertaken with disastrous results

• Duplicitous Paths -With Microsoft technologies there are often several development paths to achieve a solution and SharePoint is no exception. But despite the seeming logical and extensible nature of a path, a satisfactory result may not be achieved. As this stage of SharePoint’s maturity, many likely development paths often come to a dead-end through no fault of the logic of the developer or their approach, but because of a SharePoint bug, anomaly or undocumented feature.

• SharePoint is only early into SPK release lifecycle and the many CUs (cumulative updates) are a must

• Development Environment – a suitable development environment is required which ideally should replicate Production conditions which may involve topologies of web farms, index servers and SQL clusters, Active Directory and Exchange servers. Few environments provide that level of development platform support, although virtualization is closing the gap. This means that SharePoint development projects do not always travel well to the Production environment, even with Solutions and Packages, delivering unforeseen results

SharePoint Production environment topologies can be complex, and are not often replicated in development, with the result that SharePoint solutions do not always travel well when deployed

Metadata Intelligence
The leveraging of metadata is one of the most underrated areas in SharePoint yet it is invaluable to your business and information management strategy.

For example the implementation of an information management system, incorporating governance and compliance built on SharePoint’s metadata and workflows functionalality, is a logical application for the platform.

Metadata support allows you to attach keywords that can be used to provide meaning about the content that is loaded, assisting in categorization, search and aggregation. It is also a key to ensuring the alignment of content and business processes to your governance, compliance and information management plans, by being linked back to your associated business goals and objectives.

SharePoint OOTB can leverage the Properties page in all Office documents where metadata for author, title, subject, status, keywords, etc., can be entered. In addition SharePoint has a whole metadata infrastructure that allows you to create, customize and manage your own metadata infrastructure.

This SharePoint metadata, called content types, have particular value in that they can also be used to trigger actions and workflows. For example when a document is loaded into a library, special processes can be triggered based on its metadata.

Let’s consider some of the implications of even a basic implementation of SharePoint’s metadata on a large scale. (For ease of discussion I will use the term metadata rather than content type.)

Note the message ‘You MUST fill out any required properties’

• Mandated Metadata – It seems like a good idea to allocate metadata to all your SharePoint content because of the associated benefits. So the next obvious step is to mandate metadata i.e. ensure every piece of content that is loaded into SharePoint must be assigned relevant metadata.

• To add metadata to a single document may only take a minute but scale that out across your business to every staff member and to every piece of content, and a sizeable amount of time will be taken up assigning metadata that will erode overall business productivity

• Dirty Metadata – One way to overcome this ‘mandated metadata productivity hit’ is to make the metadata faster and more intuitive to allocate. This can be done by providing metadata in the form of dynamic lists, as opposed to free text fields.

• But even when presented with an easy or ’smart’ pick list, staff may not know what metadata value to assign, or be confronted with too many choices. For example combinations of departments, business units, and projects could produce an overwhelming list of options.

• ‘Time poor’ information workers don’t want to stop their task to find out what metadata should be allocated or go through the process to create a new category. This can lead to erroneous values being assigned just to expedite content loading.

• In some cases there may be purposeful erroneous metadata provided, to obscure business activities such as fraud

• Legacy Metadata – Most discussions of SharePoint benefits focus on ‘greenfields’ SharePoint implementations but all businesses have legacy fileshares, content and systems that can be essential to migrate into SharePoint.

• While it is possible to programmatically assign metadata during migration, in reality it is not worthwhile to do so on any scale unless a metadata or similar policy has previously been in place.

• Consider trying to allocate even simple metadata such as author, document title and version if these are not already associated with the content: fileshare locations by nature provide multi-author storage so authors are not differentiated from PAs, etc.; duplicates and drafts with the same or entirely different titles are common confounding categorization and titling; latest date may not reflect a file master version. These are just some of the issues.

This much used image helps demonstrate the typical nature of the information and content in most businesses fileshares. Determining, on any scale, the author, title, keywords of this information is in reality an impossible task.Thus much legacy information comes into SharePoint sans the metadata that could be vital to an understanding of the material, and assist with the information management of the business

• Metadata Meaning -Metadata seeks to give meaning to content beyond that given by a file name.

• How many metadata properties might be needed to adequately indicate the meaning of a document that spans many diverse areas and topics? Free text metadata would allow more comprehensive explanation but we are back to the ‘productivity hit’ issue of the time taken to enter all the metadata.

• Meaning may also differ from different perspectives. How would the allocation of ‘definitive meaning’ to files and content be achieved? How would different perspectives and synonyms be handled where different metadata words and terms are used for similar concepts?

• Consider the issue of trying to assign comprehensive metadata to all the images, video, audio and VoIP content in your business. How much resource would be required? Would the metadata be accurate and definitive using any current methods?

• In terms of scale, how would the process of allocating metadata relating to every email be handled?

• What about metadata relationships between content, such as between documents and documents, images, emails, projects, departments, people, roles? Would this type of meaning and understanding gained provide any useful insight into your business and be of value?

These are issues that may currently seem most poignant to the legal profession and e-Discovery but are increasing in import for all businesses.

Metadata that provides meaning as described above, including that for unstructured content, can provide valuable insight into business operations providing a basis for real business intelligence, way beyond that of the traditional BI with just numbers and charts in a spreadsheet.

Conclusion
SharePoint is a good information management and general office productivity platform. However it has its issues that are best to be aware of and managed, rather than just ‘let loose’ in your business, preventing it from becoming a ‘black hole’.

In the context of the issues discussed above:

The Good is that SharePoint will, in line with the Pareto principle, OOTB meet roughly 80% of your general information management needs and in the process provide a stable scalable platform.

Good also is that assistance with SharePoint development, both in terms of available tools and developer experience, is increasing at a rapid rate. At the time of writing this article, a quick search of www.CodePlex.com returned nearly 700 ‘open source’ projects providing explanation and assistance on SharePoint development. Countless blog articles by SharePoint gurus likewise provide commentary on numerous SharePoint solutions.

The Bad is that if you want SharePoint, then SQL Server and its associated overheads are unavoidable.

Bad also is that with regard to going beyond the metadata basics, there is no solution available within SharePoint that address the issues discussed. However this is not so bad as it seems as there is no solution available within any of the current generation of similar platforms.

How to get more meaning from business information and unstructured content is the next major conundrum facing business managers today. However I believe this type of metadata support and analysis is one of the most exciting challenges, and will be the basis for the next level of real business intelligence and competitive business advantage.

Next generation tools, providing definitive meaning to structured and unstructured information and a full understanding of information relationships, are just starting to enter the mainstream and gain more attention. These tools can work with SharePoint as well as hundreds of other information sources and hold the promise of providing an incredibly comprehensive understanding and powerful information management tool for your business.

I will be looking at this next-generation solution in more detail shortly.