SharePoint Magazine » interface

Incorporating a Master Page into a SharePoint site definition

ggalipeau — Mon, 08 Dec 2008 14:35:37 +0000

Introduction

This is the sixth article in a six-part series on ASP.NET Master Pages and SharePoint.

Introduction to Master Pages.
Examining the out of the box Master Pages in SharePoint.
Developing a custom Master Page for SharePoint.
Deploy a Master Page for a production ready system.
Customizing the Application.master Page.
Incorporating a Master Page into a SharePoint site definition

In the previous article of this series we discussed how to build a customized master page and deploy it with a feature. However, the manual process of deploying the feature to each server in the farm and activating it isn’t as seamless as most developers would like. It would be nice to have the features automatically activate themselves when a new site in SharePoint is created from a template.

This article will discuss the creation of SharePoint site definitions, which are templates for SharePoint sites that will automatically create the master page feature on the site when created.

Site Definition

Site Definitions are templates that developers can build in SharePoint. You might also hear the term Site Template in SharePoint, which is a customization of a Site Definition. In order to clear up any confusion on these two terms, you can just think of them like this:

Site Definitions – SharePoint template that developers can create to manipulate how a site should be created. These are stored in the 12 hive of SharePoint and are referenced during site creation.

Site Templates – Customization of a Site Definition that administrators of the SharePoint site can create from the UI screens. These are stored in the content database and these are not meant for developers to manipulate.

When you create a SharePoint site and use one of the out of the box templates, you are actually using a Site Definition that Microsoft created and shipped with the SharePoint product. For more information about how SharePoint constructs the out of the box Site Definitions, please refer to the second article in this series: Examining the out of the box Master Pages in SharePoint .

WSP Solutions

In the fourth article of this series: Deploy a Master Page for a production ready system , we discussed the manual process of creating a feature, placing it in the 12 hive and deploying it with stsadm commands. While that is a valid process to deploy to SharePoint, it is a very manual process. Also, we are now bringing another factor into our project (i.e.: the Site Definition). So, we could manually deploy the site definition in addition to the feature, but now we are starting to have a lot of manual steps. Plus, if we have a large SharePoint farm, we are going to have to repeat these steps on each server.

Thus, SharePoint created a way to incorporate all your features, site definitions, etc… into one deployable solution. These solutions are called WSP files. WSP files are basically CAB files with an extension SharePoint understands so it can deploy to a SharePoint farm. These solutions automatically get deployed to all appropriate servers in the farm. Thus, you only have to install the WSP against the SharePoint web application in order to install all your features, site definitions, etc…

What are we showing in this article?

In this article we are going to show how to create a Feature, Site Definition and WSP solution.

The Feature will deploy the master page to the master page gallery of our site, just like in article four of this series: Deploy a Master Page for a production ready system.

The Site Definition will create the template for our site. This template tells SharePoint which aspx pages to create, which web zones to create, what features to activate, etc…

Lastly, the WSP solution will wrap everything up into one deployable package.

One question that undoubtedly comes up is “why are you incorporating the feature into the site definition instead of using feature stapling?” Feature stapling is the process of creating features and stapling them to Site Definitions that already exist. For example: let’s say you created a web part in a feature and you wanted that feature to be activated on all sites created with the WSS team site definition that comes out of the box in SharePoint. However, you want to keep all the other parts of that site definition intact, all you want to do is add one more feature onto it. With feature stapling you can create a feature and tell it to attach itself to pre-existing site definitions (even the ones that come out of the box in SharePoint).

The decision to use feature stapling versus custom site definitions has to deal with the amount of customizations you need to make. There might be times when you want to use feature stapling instead of brand new site definitions. However, in a lot of cases you need to make changes that go above and beyond just the features in a site definition. Thus, it is good to create custom site definitions to manipulate these functionalities. Plus, custom site definitions activate the features when the site collection is getting created, while feature stapling activates the features after the site has been created. This timing is important to understand in some complicated situations.

So, for this particular demo I am going to activate my master page feature in a custom site definition to show the process. But, I am not saying this is the best way. If you already have a site created and just need to deploy the master page feature, you might want to look into feature stapling as an option.

What tools are we going to use?

One of the more difficult things to understand when you first learn how to create WSP solutions is the install files that are needed to build the WSP solution. These files are called the manifest and the ddf files. These files basically explain to SharePoint what features, site definitions, etc… is part of your solution so that SharePoint can install the solution to the web application.

Since these files are difficult to understand, there are a lot of third party tools available that will build the manifest and ddf files for you. These tools are usually extensions to Visual Studio. Microsoft built one called Visual Studio Extensions for WSS. Plus, there are many open source ones on CodePlex including STSDEV, SPBuilder, WSPBuilder, etc… Each of these tools provides a different approach to building SharePoint solutions.

My personal favorite is WSPBuilder. The way this product works is it mimics the 12 hive in a Visual Studio solution. As long as your Visual Studio solution has the same folder structure as the 12 hive, it will build the correct files to create the WSP solution. This is very powerful and flexible. I like this over the Visual Studio Extensions for WSS because I feel that I am restricted in what I can do with Visual Studio Extensions for WSS.

The choice of the solution builder tools you want to use is a personal choice. For this article I am going to work with the WSPBuilder add-in for Visual Studio.

Here is an example of how WSPBuilder works. Notice that the servers 12 hive matches the Visual Studio solution:

Servers 12 hive

Visual Studio Solution

As long as the 12 hive matches, you can right click on your project and use the WSP Builder menu options to build your WSP file.

Also, when you are developing on a SharePoint server, you can right click on your project and use the WSPBuilder menu options to deploy your WSP file to the local server. This will deploy it to all web applications on your development machine.

Thus, when you are developing, you should be developing on a SharePoint server. Virtual machines such as VPC or VMWare are great for this. Then, every time you make a change you can just go to the deploy menu option of the WSPBuilder menu. After you are finished developing, you can build the WSP file using the menu options and it will physically place the WSP file in your projects folder. At this point you can copy that WSP file to the production machine and deploy it with stsadm commands.

Walkthrough

Create The Structure

Start Visual Studio and choose a new WSPBuilder project.

Create a folder structure for the solution that matches the folder structure for the SharePoint 12 hive (this is really important – WSPBuilder works on the premise that the Visual Studio solution matches the structure of the SharePoint 12 hive). For now we are just going to create the following folders:
- Template – subfolder under 12 that holds the other subfolders
- 1033 – holds the xml files that will tell SharePoint about our new site definitions
- Features – holds the new features we are going to build (i.e.: lists, webparts, master pages, etc…)
- Layouts – holds our reference files (i.e.: images, css, js, etc…)
- Site Templates – holds our site definitions

Create a Site Definition

Create a new site definition based off the team site definition in SharePoint.
Copy the team site definition folder from the 12 hive:

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SiteTemplates\sts

- Paste the folder in the Visual Studio solutions SiteTemplates folder.
- Rename the sts folder to something unique (DemoCompanySiteDef for this example)
- Delete the defaultdws.aspx file because we aren’t going to use it for this example.

- Open the default.aspx file and change the master page to a new master page. Call this custom.master (yes, I know that this is not the name of our master page. We are going to configure SharePoint to do a string replacement of custom.master with DemoCompany.master in later steps). We will create this master page in the next step.

<%@ Page language=”C#” MasterPageFile=”~masterurl/custom.master”

Modify the Onet.xml file of the Site Definition for initial preparation

Open the Onet.xml file (located in the xml folder) and clean it up for our simple site definition.

The Onet.xml file is the file that tells SharePoint how to create the site definition. Whenever you create a new site in SharePoint it reads from the Onet.xml file to determine how to create the site. It is important to note that SharePoint is reading from the Onet.xml file when the site is being created. So, if you already have created a site, you can’t go back to the Onet.xml file, make changes, and expect it to show up on your already created sites.

Scroll down to the Configurations XML nodes and remove 2 configurations:
- ID=1 – Blank
- ID-2 – DWS

Be very careful not to remove ID=0 – Default, because this is the configuration we are going to use for this site definition. When removing the other two get rid of the whole xml tag as well as everything within their xml tag.

Scroll down to the Modules XML nodes and remove 2 modules:
- DefaultBlank
- DWS

Once again, be very careful not to remove the Default module because this is the module we are going to use for this site definition. When removing the other two get rid of the whole xml tag as well as everything within their xml tag.

Create the new master page feature

The master page will be created as a feature. The reason for this is that if the master page is in a feature the master page can be added to the master page gallery of SharePoint. This allows you to set out-of-the-box pages to the master page or your own custom site definitions. It also allows you to turn on or turn off the master page features on specific sites. Basically, having it in a feature (rather than directly in the site definition or in the layouts folder) gives you much more flexibility.

Create a new folder under “FEATURES” and call it something unique for your master page. For this example we will call it DemoCompanyMasterPage.
Create a folder under the DemoCompanyMasterPage called “MasterPages”.
- Copy the general SharePoint master page from the 12 hive: C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\GLOBAL\default.master
- Paste the file under the “MasterPages” folder in the Visual Studio solution.
- Rename the file to something unique (DemoCompany.master for this example).
Create two xml files under the DemoCompanyMasterPage folder.
- feature.xml – this will give a unique GUID to the feature (which we will reference later). This file also gives some important information about the Scope of the feature. Lastly, this file tells SharePoint where all the other files that make up this feature lives. This file must be called feature.xml (this is because SharePoint will read all files called feature.xml from the Features folder).

Scope=”Site” Version=”1.0.0.0″ Hidden=”FALSE” DefaultResourceFile=”core” xmlns=”http://schemas.microsoft.com/sharepoint/”
Description=”This Feature contains the demo master page”>

Please note that the Scope=”Site”. This is very important. This is telling SharePoint that this feature is for the site collection. If that Scope=”Web” then it would be telling SharePoint that this feature is for an individual site.

elements.xml – this file is telling SharePoint to place the new master page into the master page gallery. This file does not have to be called elements.xml (unlike the feature file which has to be called feature.xml). It just has to match the name located in the “ElementManifest” of the feature.xml file.

Modify Onet.xml file of the Site Definition

We are modifying the Onet.xml file of the Site Definition in order to tell this Site Definition to activate our Master Page feature when the site is created and to use our custom master page with the dynamic url replacement technique.

Dynamic Token URL replacement

To learn more about Dynamic URL replacement please refer to the second article in this series: Examining the out of the box SharePoint Master Pages.

Open the onet.xml file in Visual Studio (located at SiteTemplates/DemoCompanySiteDef/xml).
Find the xml node that says Configuration ID=0 Name=”Default”.
Add in a reference for the new master page:

MasterUrl=”~SiteCollection/_catalogs/masterpage/DemoCompany.master”>

Note: SharePoint will do a string replacement of any aspx page with ~masterurl/custom.master with the reference in CustomMasterUrl. And, it will replace any aspx page with ~masterurl/default.master with the reference in MasterUrl.

Activate the Feature

Find the SiteFeatures XML node.
Add a new Feature node in there that matches the GUID of the feature.xml file we created for our master page.

This section of the onet.xml tells the site to activate this particular feature on this particular site when it is being created. The reason it goes into the SiteFeatures (instead of the WebFeatures) is because this particular feature was scoped for a Site Collection. If the feature was scoped for an individual site then it would go in the WebFeatures.

Create the WebTemp*.xml file

WebTemp*.xml files tell SharePoint about site definitions. When a new site is being created in SharePoint the “Select Templates” list box is populated by anything it can find in the WebTemp*.xml files. There can be multiple of these files in SharePoint as long as they are in the 12/Template/1033/XML folder. SharePoint will basically read this folder for any file that starts with the word WebTemp.

Add a new folder under 12/Template/1033 in our Visual Studio solution called XML.
Add a new file to the 12/Template/1033/XML folder in our Visual Studio solution. This file must start with WebTemp. We will call this file WebTempDemoCompany.xml.

Add in the proper xml to tell SharePoint about our new site definition

A few things to note:

The “Name” must match the site definition folder exactly. We used the name DemoCompanySiteDef. This matches our site definition name exactly.
The Configuration ID must match the configuration ID in our onet.xml file of our Site Definition. We used configuration id 0.
The display category will create a new tab on the new site screen in SharePoint.
The ID has to be unique and SharePoint has already reserved certain IDs. To be safe use an ID over 10000 and make sure it is unique.

Use WSP Builder to build and deploy the solution

Now, at this point (in a regular SharePoint solution) we would create manifest.xml and ddf files. These are files SharePoint needs to install the solution properly. However, since we are using WSPBuilder, it will do that for us.

Right click the project and click WSPBuilder – Build

Verify your Web Application is created on your local SharePoint environment. Note: the Site Collection should not be created at this point, just the Web Application. And, you only need to do this once.
If the build is successful, right click the project and click WSPBuilder – Deploy.

This is needed because the build command in WSPBuilder just builds a wsp file. This file can be sent to other machines and you can use stsadm scripts to install it on other SharePoint farms. However, in development, we need a quick way to deploy it to the current SharePoint farm. The WSPBuilder – Deploy command will deploy the wsp file on all the Web Applications in the local SharePoint farm.

Use the Site Definition

Open up SharePoint Central Administration and create a new Site Collection with the site definition we just created.

Now you can go back and modify the master page or default.aspx page as you wish. Just be careful not to mess with the ContentPlaceHolders too much. SharePoint reserve these for certain actions on some of its internal pages. After every change you make you can rebuild by clicking WSPBuilder – Build and update the solution by clicking WSPBuilder – Deploy. Then you can just go to the website and see your changes. Just be aware that everytime you deploy WSPBuilder does an app pool recycle (to ensure the changes get pushed out through IIS), so you might have to wait a few seconds to see your changes (during that few seconds the site could say Service Unavailable).

It is also important to note that the first time you deploy with WSPBuilder it does a true SharePoint install. However, every subsequent time it does a SharePoint solution update. Solution updates work for most changes, however, there are some things it can’t do. So, if you run into errors it is sometimes good to do WSPBuilder – Uninstall and then WSPBuilder-Deploy to do a real install in certain situations when you are making changes.

If you are curious to what a SharePoint solution is doing, then just check out the files in the 12 hive:

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12

You will notice that all the files we created in our Visual Studio solution have been “pushed” down to the folders in the 12 hive. The WSP knows to “push” these files there. The WSP also knows to tell SharePoint that these files exist. This is exactly how SharePoint develops it’s out of the box site definitions.

Side Notes:

1. Any file you place in the Layouts folder can be reference in the aspx pages as “/_layouts/{filename}”. I usually recommend placing another folder under Layouts in the Visual Studio solution (just because it makes it easy to find later – you can call this folder the name of your company or the name of your project or anything that is obvious that it contains your custom reference files). Then you can reference everything in there by “/_layouts/{folder}/{filename}”. An example of this is css files in the master page. This is possible because SharePoint sets an IIS path up to the Layouts folder in the 12 hive and calls it _layouts.

2. If you want to create a custom webpart with WSPBuilder just click Add – new item… WSPBuilder – WebPartFeature. This will create the feature folder, for your webpart, and the .cs class for your webpart. This way you can go directly into coding and not have to worry about the Feature creation. Once you are done coding the webpart just make sure it is in the appropriate scope of your onet.xml file (SiteFeatures or WebFeatures) and it will become available to you in your webpart gallery. Then all you have to do is worry about your dll being installed in the Bin or GAC. If you choose GAC I recommend post-build scripts to move it.

3. Do not remove Content Place Holders from your master page if you start doing customizations. The reason for this is if you assign other files in SharePoint (such as list pages) to use your master pages, they will assume certain Content Place Holders exist. If they are not there, your site will just break.

Customizing the user experience: Overview of the default interface (Part 1 of 6)

furuknap — Thu, 09 Oct 2008 01:00:12 +0000

Have you ever wondered how the built in interface actually works? Why is a list item shown the way it is? How are the input forms built and how can you change how they work and look?

This series will introduce you to a wide variety of customization options not usually covered in books or online articles.

Most articles will have rather high pre-requisites. Basically you should be very familiar with ASP.Net, SharePoint and CAML. You should, for instance, understand very well how master pages and content placeholders work, you must absolutely be able to create a feature from scratch and you should have some experience with working with content types.

So, if you are up for it, here is an article outline for the six parts:

Part 1: Overview of the default SharePoint interface from a technical point of view

In the first article we will look at how the default SharePoint interface is built. We will look at a site, going from top-down, explore some of the the default lists, the fields used to create the basic field types, which content types are available, and how list forms are rendered.

Part 2: Modifying the default experience

This article will show you which options are available for you to modify and improve the default setup. Learn how to override the default rendering of fields or forms without voiding your supported state.

Part 3: Lists and custom list forms

The third article will cover the basics of customizing lists using different views, custom list forms, and fields.

Part 4: Content types user interface

The next article will explore how you can utilize content types to display different input forms and display forms.

Part 5: Custom fields deep dive

Ever wanted to create a new field type? SharePoint enables you to do this and it is a very powerful tool for customizing the user experience.

Part 6: Fast track to feature generation

Writing custom lists with content types by hand can take a massive amount of time. In the final installment I will share with you some tools and techniques that makes list, field, and content type generation very fast.

So, what are your waiting for? Let’s get going…

Disclaimer

Before we go any further, be advised that any changes you make to files in the 12-hive may void your warranty and cause damage to your SharePoint installation. I will come back to changes you can make and what you cannot change in the next article, but for now, feel free to experiment only in your non-supportable lab environment. I would ask you to take backups of anything you change, but we both know you won’t do that anyway, so let’s cut the crap and just admit that we are and can be renegade cowboys, or cowgirls, as long as we do so in a lab where we will not be needing Microsoft’s rescue if something goes wrong.

Also, I won’t be covering everything. Simply not enough time, and I want to keep the article to at least below a day’s worth of reading.

That’s it.

Nothing else to see here, move along.

Overview of the default SharePoint interface from a technical point of view

Actually, the term overview may be a bit inaccurate. Be aware that we will go deep into some topics. However, each of the topics we will investigate are worthy of books and books of even deeper diving. While it may seem that this article covers a lot, it does not by far cover everything, but only gives an, you guess it, overview of a whole range of aspects.

Let’s start our journey of exploration right away and get to know our 12-hive a bit better. You know what the 12-hive is, right? If not, feel free to read up on it and come back to the group later. (Ok, I just wanted to sound a bit arrogant, the 12-hive is just another name of the file structure that usually resides in c:\Program Files\Common Files\Microsoft Shareed\web server extentions\12)

Oh, and by the way, from now on I will address the 12-hive as [12], so whenever you see [12] you should know where to go.

Sites

We want to know of what a site is actually made. How are the different lists made, how are the .aspx files created and from where do they come, why do you get all those _catalogs folders, and a lot of other things. For this I will use the Basic Meeting workspace site definition and really dive deep into how it is created. In the following articles we will use our knowledge to create new lists, pages, and definitions, but we need to know what is going on first.

The [12]\TEMPLATE\1033-folder, or the numbered folder that corresponds to the language pack you have installed, contains a few different things related to the language pack in question. For starters you will find your webtemp.xml files in the XML folder there. The ‘temp’ part of that filename is short for template, not temporary, by the way.

The webtemp.xml file, or files if you have created your own site templates, contain the templates available for site and site collection creation. The webtemp.xml is actually just the list of templates and some minor configuration data, the bulk of the site template definition is actually in the SiteTemplates folder in the [12]\TEMPLATE folder. We will get there later.

webtemp.xml

If you want to create your own site templates you need to add at least one file here, called webtempXXXXX.xml, where XXXXX is something of your choosing. You cannot modify the webtemp.xml file that ships with your language pack or you will void warranty. A notable exception to this is to change the hidden attribute. You can use this to hide any templates you do not want users to see during the site collection creation process.

If you open the default webtemp.xml file you will see both Template elements and one or more Configuration elements within those Template elements. The Configuration elements are the actual site definitions, and for each of these Configuration elements there must be a matching Configuration section in an onet.xml file somewhere.

Where? We will get there later, I told you! Be patient.

Look, for instance, in the Template element named MPS having an ID of 2, this being the template for the meeting workspaces. There are five different configurations there, each being a different type of meeting workspace. To properly reference a specific site definition you need both the name section and the ID, in the format of “NAME#ConfigurationID” . So, if you want to programmatically create a site you would use, for instance:

SPWeb newWeb = web.Webs.Add("NewMeetingSite", "My new basic meeting workspace", "My description", 1033, "MPS#0", false, false);

Now it is time to award your first bit of patience. You see, all the configurations in webtemp*.xml are very basic, but still you get a lot of stuff happening when you create a site using that configuration. There must be more stored somewhere, right?

Let’s head out of the 1033-folder and to [12]\Template\SiteTemplates for a while.

Notice anything in particular? Three are folders there, true, but what are they named? Exactly the same as the names of the site templates in webtemp*.xml! There must be a connection in there, we must press on if we are to discover the true secret of the site definitions.

Open the MPS folder. Now we are getting somewhere. Look at the default.aspx file using your favorite editor. If you dissect this file you will see that it is actually the file used to render the front page of the new meeting space. Not just that, but changes made to this file will alter any Meeting Workspace in your entire site collection. I can feel the power rushing to my head. Delete it an no one will be able to hold a meeting any longer! Muhahaha! (Cue music from Stargate SG-1 or your favorite evil ruler movie scene) Obey me, office slaves, kneel before your master! I rule your meeting life from this day and until eternity! Muhahaha!

Oh, sorry about that.

Actually, this is only partially cool. If you modify a page in SharePoint, for instance the default.aspx page of a meeting workspace, your page becomes ‘Customized’ meaning it is no longer the same as it was when it was created from the site definition. In practice, what happens when you customize a page is that SharePoint retrieves the page from the site definition, ie the page we just deleted (no, you really shouldn’t delete it, unlimited power does not come from getting yelled at from your users). Then it stores a copy of that file in the database. When you then make changes to the file you only change the file stored in the database, not the actual file on the file system.

If you use a function to restore the file to the site definition, for instance from the web interface under site settings or using a SharePoint compatible editor such as SharePoint Designer, you just say to SharePoint that you do no longer want to use the file stored in the database but want to use the one on the file system, which you should have restored from backup now if you deleted it earlier.

You know, that backup…

Oh, bother…

We still need to find out more about our site, and the really good stuff is still ahead.

onet.xml

Back in your MPS folder you will also find a folder named XML, and inside you will find a file called ONET.XML. onet.xml is the file that actually holds the details of the different configurations, amongst other things. It follows the wss.xsd schema so you can get intellisense if you are using Visual Studio as well and this is a real life saver when you start writing your own site definitions. You can check out the instructions for getting intellisense in onet.xml in my blog at http://furuknap.blogspot.com/2008/04/intellisense-in-onet.html.

If you close all the first-level nodes in Visual Studio you can get a quick overview of what is inside the Project root node. In the MPS onet.xml you will find NavBars, ListTemplates, DocumentTemplates, Configurations, Modules, Components, and ServerEmailFooter.

The NavBars element contain the NavBars, go figure. NavBars are the navigational elements of a site, such as the quick launch and the top navigation bar of a team site. The navbar elements are the places where you define navbars and create and modify the links that appear in each of these navbars. More on that in a later article.

The ListTemplates section contains the list templates that will be made available with the site definition. If you go to a newly created Basic Meeting workspace and click Create from the site actions menu you will see some additional custom lists that are not available in, for instance, the Team Site template. The ListTemplates section describes the basics of these custom lists.

Note that, as for the webtemp.xml and the configuration section there, the ListTemplates are only the list of listtemplates, not the actual list templates. Hang on a bit and we’ll dig even deeper to find the detailed list templates, just like we found onet.xml from our webtemp.xml file.

The DocumentTemplates section contains the document templates you can chose when you create a new document library. In your own onet.xml you can add document templates here and they will appear for all document libraries for all the configurations in your site template.

The next section, Configurations, is where the brunt of the work is done in onet.xml. Notice the ID of each Configuration element? These IDs correspond to the configuration elements of webtemp.xml. Our code snippet above created a site based on the site definition MPS#0. We know that the MPS refers to the onet.xml file in the MPS folder under SiteTemplate and now we find what the 0 means.

Open the Configuration element with ID 0 and name Basic. Note that it uses a CustomMasterUrl attribute that refers to “_catalogs/masterpage/MWSDefault.master”. We will explore how that file is actually created when we come to the modules section.

The Lists element and its children List elements tells SharePoint that we want lists of a certain type. Remember the ListTemplates section? The Type of the List elements refers to the ListTemplate of the ListTemplates section.

ListTemplates may also exist in other places than in the onet.xml file. If you look a bit further down you see that the Lists section contains a List of type 101, and we do not have a 101 type in out ListTemplates section. So, where is list 101 located?

Well, first of all, lists with numbers in the 100s, 200s and 300s are so-called built-in list templates. 101 happens to refer to the generic Document Library list template. You can find a complete list of the built-in template types at http://msdn.microsoft.com/en-us/library/ms462947.aspx. But where is it?

I’ll keep you on the tip of your seats a bit longer. We’ll get there soon, I promise.

Next you will find a Modules section with two Module elements named BasicHome and MWSMasterPage. These module elements tells SharePoint what files, for some reason termed modules, should be provisioned with the site. We only see the name of each module here, though, not which files are actually copied. The modules are defined in the Modules section of the root element so we will get there soon as well.

You will also find a SiteFeatures and a WebFeatures section. Each section refers to features that are site or web scoped, respectively. These section defines which features should be activated with the site definition. If can be confusing to know which features are actually activated since only their Guid reference is used so you would need to search in the Features folder of your [12]\Template folder to find the correct names. The two features activated with MPS#0 are the three-state workflow (Guid FDE5D850-671E-4143-950A-87B473922DC7) and team collaboration feature (Guid 00BFEA71-4EA5-48D4-A4AD-7EA5C011ABE5).

There is one more element of the configuration section that is not included in any of the MPS definitions, and it can be extremely useful so I will include it here. I am talking about the ExecuteUrl element. The ExecuteUrl is a URL that will be launched when the site is created. This can be incredibly powerful because it allows you to perform additional tasks such as further configuring your site, inviting users, giving users intructions on how to manage their new site, or any other task that you may want to accomplish. You can redirect the user to the Create new content page, for instance, or you can create your own configuration page tailored to your needs.

The ExecuteUrl is only launched when a user creates the site through the web interface, though, not when created by other means such as through custom code. Still, it can be extremely useful for creating a richer user experience. Use it. I do. And you want to be like me, right?

Don’t answer that.

Moving along, we can now close the Configuration section and look at the Modules section.

The modules are where many of the files for our site are defined. Remember we had two modules in our MPS#0 definition, named BasicHome and MWSMasterPage? This is where we find out what these two modules actually comprise. The reason we only needed to include the names of the modules in our configuration is that the rest is defined here.

A module can comprise multiple files, which may be why they are not simply called ‘Files’, and when a module is used by a site configuration all the files of the module are provisioned according to the module definition.

Open the BasicHome module. Notice that there is a Url and a path attribute. The Url attribute says where the files should be stored on the site. For BasicHome it is empty, but it is defined for the actual files that are stored in the module.

The path attribute of the module tells SharePoint where to find the file. So, if it is empty, where is the file? The path is actually relative to the feature or the site definition, so an empty path tells SharePoint that the file is in the root folder of the feature or the site definition. In our case that would be in [12]\TEMPLATE\SiteTemplates\MPS.

If you look at the only File in this module, default.aspx, you will see that it has a URL but no path. The rule here is that if no path is set then Url will be used as path, so if your file has a physical name that is the same as you want it to be on the site then you can skip the path attribute.

See now how SharePoint mapped the default.aspx of our site to the physical file store in the MPS root, and why editing or deleting that file (no, you shouldn’t this time either) will modify all the sites that use this definition? Pretty cool, eh?

Now, contained in the default.aspx File element you will also find a few View elements and a NavBarPage element. All of these are useful. The View elements puts webpart with views of our lists into our file. Each view element has a list, which refers to one of the lists was created in the Lists section of the configuration, a BaseViewId which states which view will be displayed for the list as well as what webpartzone should host that view. There can also be a WebPart order if you have multiple views in the same webpartzone. We will get back to how the lists and their views are defined later. Using the View elements you can thus populate your webpart page with webparts during the site creation.

The NavBarPage is also useful. NavBarPages tells SharePoint where in the navbar this page should be placed, as well as the name which should be used when linking to that page. Note that several of the navigational options available in a site definition does not behave as expected. In fact, the NavBarHome attribute of a file in a module doesn’t work at all. An exhaustive investigation is beyond the scope of this article, though, and I must leave some goodies for my book

Note that the name itself is stored in a resource file. That’s another show.

There is one more very useful thing I would like to mention regarding the File element, and that is the AllUsersWebPart element. It is not present in this module, or even in the MPS onet.xml at all, but look at the STS onet.xml, located at [12]\TEMPLATE\SiteTemplates\STS\XML\onet.xml, under the module named Default:

Ever been annoyed by the SharePoint logo that appears on every new team site? Well, there’s your culprit. Inside the AllUsersWebPart is an XML CDATA containing the straight-up webpart code to add a regular ImageWebPart webpart containing that image.

You can use this to your advantage, though, to add any webpart, not just views of lists, to your page. I will share with you some simple tricks for working with this in the last article of this series. And you know that when I ask you to wait so long it must be good.

The last thing we should explore in the onet.xml file for now is the MWSMasterPage module. It is a bit different from the default.aspx module, but there are some notable differences.

The first thing that may catch your eye is that there is a SetupPath pointing to “global”. Now, for the BasicHome module we learned that path refers to the path in the site definition, but we do not have any global folder, do we? How will we find the MWSdefault.master file in the only File element of this module?

The key here is that this is not Path, but SetupPath. Setup refers to the SharePoint template folder, otherwise known as [12]\TEMPLATE, so SetupPath is not relative to the site definition folder but rather the [12]\Template folder. If you go there now you will see a global folder, and inside there, guess what, there is a file called MWSdefault.master.

The module element also has a Url attribute with a value of _catalogs/masterpage. This simply tells SharePoint where the files in this module should be stored. However, if the folder does not exist you will not get your file provisioned. How, then, does SharePoint know how to create the _catalogs/masterpage directory? The plot thickens. More on that later.

The file for MWSdefault.master contains an attribute called Type with a value of GhostableInLibrary. This tells SharePoint that the file should be cached in a library and that you should be able to work with the file as you would any other file in a library. If you go to the masterpage gallery of a site you will notice that each masterpage is a file like any other in a library. This is because they are provisioned as GhostableInLibrary rather than just Ghostable. A Ghostable file, like default.aspx, does not reside in a library. GhostableInLibrary files have version history, created by and modified by properties, and any other column that the library provides. A simply Ghostable file one does not. A GhostableInLibrary file can be checked out, approved, deleted, and anything else a library file can. A simply Ghostable file can not. Bascially, if you want to work with a module provisioned file through the web interface, use GhostableInLibrary.

The last two sections of the onet.xml file are Components and ServerEmailFooter. The latter is self-explanatory, but the former is quite strange and potentially incredibly cool. It contains a FileDialogPostProcessor with an ID. What this actually is is a class that is used to modify the file open and save dialogs on a document library. You know, when you save a file from Office directly to a SharePoint site you get a cool web interface in the Save dialog? In the FileDialogPostProcessor you can declare a class that will modify that dialog web interface. Pretty obscure, even for this series, but I may cover it in a later article if there is sufficient interest.

Done with onet.xml, time for some patience reward again.

Lists

Before we really dive into the list template goodies I want to answer the question about where the list with type 101, the Document Library, resides.

The Document Library list type is actually a feature, located in the [12]\TEMPLATE\Features\DocumentLibrary folder. Go ahead and open that folder. The contents of that folder is the actual SharePoint feature. Open the feature.xml file at the root of the DocumentLibrary folder.

feature.xml and element manifests

Every feature that actually does something has at minimum two files, the feature definition file and one or more feature elements files. The first file, by convention called feature.xml, defines the configuration of the feature, such as the scope, ID, name, description and such.

The feature.xml does not actually do anything by itself, though, it only configures the feature and lists element files that should do the heavy lifting. In our case we have only one file, an element manifest, contained in our feature.xml. The location listed for the element manifest is relative to the root of the feature, and if you look in the root of our DocumentLibrary feature you will in fact find the file listed (ListTemplates\DocumentLibrary.xml). Open that file now and note that the documentlibrary.xml contains a ListTemplate element with a type of 101. Suddenly we now see where the list template type 101 gets created.

Still, there must be something more. I mean, where are all the columns and views and content types… We are getting closer to finding the treasure but we are not quite there yet.

Notice the name attribute of the ListTemplate element with a value of doclib. SharePoint uses the name of the template to search for an XML file called [name]\schema.xml in the root of the feature. So, it makes sense to look, in the DocumentLibrary root folder, for a folder named doclib containing a file called schema.xml. If you do there now you will find the main part of the list definition, the schema that defines almost everything about the list template. Open it now, and be prepared for the grand tour of list schemas.

schema.xml

Actually, there is not to much about the schema.xml. Sure, it looks scary because of its sheer mass, but if you break it down there are really not that many aspects to it. There is a List root element containing a MetaData element, and inside the MetaData section are only five elements that make up the entire list. Nothing scary at all.

See how tiny it is? We’ll get through that in no time at all! Actually, we will go deeper into schema.xml in the third article of this series, so for now we will just skim through to grasp the basic outline of this file.

ContentTypes

The ContentTypes section contains content types or references to content types. We will go deeper into content types defined here in the third and fourth part of the series, but for now, I want to explain the difference between ContentType and ContentTypeRef. You see the ContentTypeRef variety references a site content type while the ContentType is a content type defined in the list. I know that may sound confusing, especially since the documentation does not include anything to suggest that you can actually define a content type in a list, but you can, and the wss.xsd schema for schema.xml let’s you do so. We will come back to how this can be used to create some really powerful customization in part three.

ListForms

Next comes the Fields and the Views, but I would like to skip these for a moment and go the the Forms section. Open that now.

There are four forms in the Document Library schema.xml, and you may have heard these names before. As these are very relevant to the user experience I would like to spend a little time getting to know them. We will again come back to this in part three.

The three basic forms of a list item are DisplayForm, EditForm and NewForm, and I am going to assume you are smart enough to figure out which role each plays. What is relevant, though, is how these forms are created when we provision a list such as the document library. We will come back to that in great detail in part three, but for now, know that you can customize each of these forms just as you want. This means that you can have your default forms look and behave exactly how you want, including adding AJAX, custom field controls, visual elements, heck, you can even add a Tetris minigame half-way down the page to give your users a rest while they are editing an article.

If you look at the Form elements you find that the display and edit forms have a SetupPath attribute, as well as a Url attribute. The SetupPath works in the same way that the module SetupPath does, in that it references a file located somewhere in [12]\TEMPLATE. So, if you look in the [12]\TEMPLATE\pages directory you will find a form.aspx, again a rather plain vanilla ASPX page. This form.aspx is use to render the outline of the page used for entering, displaying and updating item data. However, note that there are no actual fields in the form.aspx file. The only thing even close to something unusual is the presence of a web part zone with an ID of Main. Strangely enough, this is also referenced in the Form element. How does all this add up?

Content types and list forms

All the forms can be configured on the list level, but what is also cool is that you can do this on the content type level as well. As I said earlier, we will explore content types in greater detail later, but the ability to connect a given form to the content type rather than where the information is stored gives you incredible control over the user experience. In fact, you have seen this in action plenty of times, but you may not have known how this actually works.

Every content type has a set of forms that matches the listforms. You may not have specified any particular form if you have written your own content types, but that only means that one is provided for you by inheritance. You see, every content type ultimately inherits from the Item content type, with an ID of 0×01. Every content type you see or create will have 0×01 in the beginning of its ID, meaning it inherits at least from the Item parent type.

The Item parent content type defines the default forms to be used for rendering the final form. By default all formtemplates are set to use a template called ListForm for rendering. I will show you where this is done later in this article.

The ListForm in turn is stored in a file called defaulttemplates.wscx, locaed in [12]\TEMPLATE\ControlTemplates. If you open that file and search for the rendering template with an ID of ListForm, you will see the actual code used to render any content type form that does not override the default forms.

Fields

Back to the fields section, and it is time to blow our minds a bit, because we are going to explore a slightly complex aspect of our list schema. Open up the fields section. You should see a lot of caml, with intermixed queries, XML, javascript, the lot.

Before you assume that this will get messy and you will never be able to create this by yourself, listen a bit longer. You see, most of the time you do not have to mess about with fields code by hand. I will show you, in part six, how to generate this code real easy. We will explore in greater depth the fields section in part three, however, you should know that you can do incredibly much with a field if you are willing to spend a little time learning the basics. In part three. And some more in part five. For now, close the entire fields section and start breathing again.

And now, for the final trick of the evening I will take ten years off your life from the shock you are about to receive. Be warned that you should notify your next of kin before reading on. Make sure you settle any scores, pay all your bills, and wear clean underwear. Behold, the most scary piece of any SharePoint list, the nightmare of many a SharePoint developer, the bane of projects small and large, I would like you to open:

The Views Section!

Still alive? No aching chest pains? No bright white light inviting you to enter?

SharePoint views are scary. Real scary. Sort of like a bunch of zombies outside your door, and all your shotgun shells are in the garage. Or, like a massive rumbling in the background when you are paddling down a river just as your only oar breaks clean off.

However, after the initial shock has resided, you will find that views are cool. And probably one of your most powerful tools for customizing the user experience. And don’t worry, we will cover a lot of the basics of views in part three. And, if you are really good, I will share with you some great techniques for creating views fast and easy in part six.

But know this, by the time we are finished exploring views you will know CAML like you know the back of your hand, or you will be in misery.

For now, close the views section. If you have loved ones, call them and let them know you are all right.

There really is only one more section of relevance in the schema.xml file, and it is not present in the document library schema, the Toolbar. However, the Toolbar element is covered to some extent in part three so we won’t go into that here.

But wait, there is more!

Ok, we have gone some depth into a site and seen how the site it built, but we haven’t by far covered everything, and a few more things must be part of our exploration.

You see, we looked at the Document Library list template that we found in our Basic Meeting Workspace, or most any other out-of-the-box site template, but what about the other list templates? It would be logical that these were also features, just like Document Library, but now matter how hard you look, you will not find a feature that defines the Master Page library, the user information list, or the site-, webpart or list template galleries. These have list types of 116, 112, 111, 113, and 114 respectively.

We know these must be provisioned from somewhere because they are lists, just like any other list, but where are they located?

Not to mention the default.master page. That seems to appear out of nowhere, as none of our out-of-the-box site templates even mentions a module containing default.master.

And how are the ‘default’ content types, such as Document and Folder actually provisioned? And the built-in fields such as Title and Modified by?

Global site definition

Well, there is one more place we have not looked yet. Actually, we were there for a short moment earlier, but we didn’t stop to see what was there. I am talking about the [12]\TEMPLATE\global directory where we found our mwsdefault.master. If you open that directory now you will indeed find something that resembles a site definition, including an XML directory containing an onet.xml file. Open that file and let your eyes feast on the secret of the remaining automatically deployed lists and files.

You will find, not only the list templates of the missing lists, but also a configuration with ID=”0″ that activates two site features with IDs of CA7BD552-10B1-4563-85B9-5ED1D39C962A and 695B6570-A48B-4A8E-8EA5-26EA7FC1D162. If you search in your [12]\TEMPLATE\features folder for these two Ids you will find that they belong to the fields and ctypes features respectively. These features in turn contain fieldswss.xml that defines all the built-in fields and ctypeswss.xml that contain the default content types.

The global site definition is applied to every site definition. So, whenever we create a new definition we get the global definition and all its goodness for free. If we were to modify that definition, every new site created from that point on would have our modifications.

But we won’t do that since that voids our warranty and supportability, right? Just thought I’d let you know.

Columns and fields

Now, the final thing I want you to see are fields and columns. The terms are often mixed up, which can lead to some confusion. Fields are basically templates for columns, with a column being the field when it is added to a list or a content type. At least that is the way I use the terms. Fields are the text, multi-line, date/time, yes/no, choice value that you can select when you add a new column to a list or a content type. Columns are instantiations of these fields, such as Title, Modified, Created by, etc.

You can think of fields in much the same way as a site or list template, and columns much in the same way as a site and list. Just as a site template, say the STS teamsite template, becomes a site that you can name almost whatever you want, a field, say Text, becomes a column that you can name almost whatever you want. The doclib, or document library list template becomes a list named Shared Documents.

The fields, even the built-in fields such as text, multi-line text, date/time, and choice, are defined through XML. This means that you can add new field types or even, if you are crazy enough, change the default types. I will show you a whole lot more on fields in part five where we will also dive into how we can create new fields that completely changes how the user enters data and thus their experience of how SharePoint works.

For now we wont go into too much detail, but I would like you to know where you can find the built-in fields. A second reason for why I will not go through this like I have done with the other files is that I would like to give you some homework. More on that in a few moments.

Wrapping your head around all this

I realize that there may be plenty of stuff here that seems overwhelming. I know, I felt the same thing when I started going deep into the SharePoint model. And to scare you a bit more, even though we have looked at a great deal in this article we have only barely scratched the surface. And even if we will dive quite deep into some of these topics in the later articles we have still only looked at the user experience. There is a lot to learn about SharePoint, in the sense that there are a lot of atoms in the universe.

I would like to offer you some advice, though, on how to cope with all this.

1. SharePoint is a platform, not a product

If you think of SharePoint more as a platform than as a finished, turnkey product it will likely be easier to understand how everything fits together. Forget the teamsite, forget the basic meeting workplace, forget the default content types, lists, and everything, and start thinking of these things as demonstrations that Microsoft has provided for you. What are they demonstrating? SharePoint as a platform. Everything you see is built on that platform, and if you start to see how they built these things, using only tools that are available to you as well, you will see how powerful SharePoint really is.

2. Get a good reference

Every good developer knows that they do not know everything. Even the most dedicated senior developers and architects cannot know every intricate detail about every attribute of every element of every schema. What you should lean, right away, is how to find information. The best developers I know are not those who know everything, but those who can find and understand information quickly. The secret tool of a good developer? Get a good reference.

The best reference for SharePoint I know is the SDK. Granted, there are a lot of missing stuff there, and some things are even plain wrong, but the majority of the SDK is good. Get the SDK today and familiarize yourself with it. You can view it online at http://msdn.microsoft.com/en-us/library/ms441339.aspx or download it from the same address. The download link is in the first section of the Community Content.

3. Learn how a feature works, don’t copy what everyone else does

There are plenty of more-or-less turnkey solutions for SharePoint out there. You can basically get most anything at least semi-done by googling and copying what someone else has already done. In my blog I intentionally write non-production code, with the notable exception of the Send Document Library file as email solution I wrote for a forum user, and I do this because I want to teach the users how something works rather than giving them something they can copy-and-paste but not understand at all.

Resist the urge to copy code. You may be productive and deliver a solution, but you wont know how or what the heck you are doing. Take the time to learn how a feature works. Why does the author add this or that, how will this be rendered to the user, where does this information go when I updated the item? In the long run you will be a far better developer and create far more stable and supportable solutions.

Homework

To prove a point, as mentioned, I would like to offer you an exercise to do before the next part of this series. Don’t worry, you wont get graded and if you skip this it won’t go on your permanent record. It is simply an exercise to help you find information and explore a bit on your own.

The built-in fields of SharePoint are located in [12]\TEMPLATE\XML, in a file called FLDTYPES.XML. Use the SDK link I showed you above and go through this file and look up the elements that make up a field. Don’t worry about the details of the rendering templates, we will get to those later. For now, understand what the elements do. We will come back to these fields in part five of this series.

Conclusion

This has been a long journey, but we have only just begun investigating how everything works. We have gone from the top down, exploring how sites are built, how lists work, how lists are rendering using information both from the list and the content type attached to that list, and a short tour by the fields and columns. Our focus has been on understanding the schemas of these elements because the schemas give us a glimpse into how a SharePoint site work.

Next week we will go into some completely new stuff when we explore techniques such as overriding rendering templates that can be used to modify the default user experience without voiding your warranty.

You can take a deep breath now. We are done. See you next time.