For your reference, these are our previous seven articles:

1. Part 1, BDC Purpose and technical architecture
2. Part 2, Application Definition File (ADF) and its Development
3. Part 3, Developing an ADF to Connect to Web Services
4. Part 4, Consuming Business Data through Web Parts and SharePoint Lists
5. Part 5, Implementing Enterprise Search with Business Data
6. Part 6, Integrating User Profiles with Business Data
7. Part 7, Working with the BDC Object Model

With that, let’s dive into BDC actions and start by asking what is an action.

What is an action?

An action is simply an operation upon a selected item. The operation can retrieve information about the selected item, or it can launch a program or Web page with the selected item as a parameter. The operation is a predefined URL link in each BDC item that enables you to act on the item, such as editing, or mapping it using Live Search as we demonstrated in Part 4.
An action is composed of a URL target where the action is to occur, and the matching parameters in the URL QueryString format. When we associate an action with an entity, we can pass parameters to that action based on the contents of one or more fields. For example, in the Live Search example in Part 4, we associate the Live Search page’s URL with a customer and provide the customer’s address to parameterize the search.

From the above, major characteristics of actions are as follows:

1. URL-based, parameter-driven
2. Actions on any entity and appear everywhere an entity goes
3. User can take action in ECB menu
4. Custom actions means custom code

A perfect example of action is the default View Profile action, as shown in Figure 1.

Figure 1  View Profile action in a Business Data Actions Web Part

Through a View Profile action we can navigate to the profile page of an entity. Clicking the View Profile action in a Business Data Actions Web Part as shown in Figure 1 above, as well as in a Business Data column, or in a Business Data List Web Part, lets us view the details of an item on that item’s profile page. Figure 2 shows that a Profile page that displays employee details.

Figure 2  A Profile page that displays employee details

What is a Profile page?

A profile page is automatically generated for each entity that has a SpecificFinder method after BDC imports an ADF file, a profile page can display all the information from a record in the BDC.  As we’ve seen in Figure 2, a profile page can display all the fields in a record for a specific employee. The default profile page contains a BDC Item Web Part for that entity.
If we put the profile page in Edit mode, there are actually two Web parts: a Business Data Item Builder and a Business Data Item that connect together. The Item Builder parses the querystring (http://win7:10001/ssp/admin/Content/Employee.aspx?EmployeeID=1) and passes the EmployeeID value to the Business Data Item, which then executes the SpecificFinder method and returns a record of Employee data.

Figure 3  Profile page components

Default Profile Page and default action

As we have seen from above section, the default action (View Profile) along with the default Profile page is automatically created when the entity has a SpecificFinder method.  To provide an ECB menu user experience, we can also add actions in a Business Data List Web part without using the Business Data Actions Web Part.

Defining an Action

There are three ways to add an action: 1) defining it in the ADF, 2) adding it in the BDC admin page after the ADF has been imported into BDC, and 3) through the BDC Object Model. Here we’re focus the first two.

Defining it in the ADF

To define an Action for the specified BDC entity, we’ll first locate the <Actions> section within the <Entities> and <Entity> hierarchy.

Figure 4  Actions is nested within the Entities/Entity in ADF

And insert the <action> portion like this one:

<Actions>
<Action Name=”Update Employee” Position=”1″ IsOpenedInNewWindow=”false” Url=”pages/editemployee.aspx?EmployeeID={0}” ImageUrl=”">
<ActionParameters>
<ActionParameter Name=”EmployeeID” Index=”0″ />
</ActionParameters>
</Action>

Adding an action in the BDC admin page

In the SSP administration page (View Application –> View Entity), the action is defined below and the URL that takes EmployeeID as a parameter: http://dpm:10001/ssp/admin/Content/Employee.aspx?EmployeeID={0}

Figure 5  Add an action the SSP Admin page

Custom Actions

We also can create custom actions for entities in BDC. Among other things, custom actions allow users to navigate to any custom page to manipulate business data.
Here are some scenarios that custom actions can do:

1. Parameterize custom search
2. Update employee in HR system
3. Customize profile page for an entity
4. Integrate BDC search into enterprise report center

We’ll be creating custom actions in the last part the article.

Figure 6 Enable View Profile action in a Business Data List Web part

This enables the ECB-style action, we will specify a Title property and add the following in the ADF:
<Entity EstimatedInstanceCount=”0″ Name=”Employee”>
<Properties>
<Property Name=”Title” Type=”System.String”>LastName</Property>
</Properties>
</Entity>

It specifies which column in the Business Data List/Related List Web part displays the Actions menu. It also specifies the name of the TypeDescriptor that corresponds to the column you want to display as the default column in Business Data lists.

In the Business Data Column scenario, we can also display the action menu and link the column to the business data profile in a Business Data list.

Figure 7  View Profile action in Business Data Column

This is default behavior OOB for a Business Data column.  When adding the column, just make sure the following two are checked:

• Display the actions menu
• Link this column to the profile page to

Figure 7  Enable the actions menu and profile page in Business Data list

These profile pages are used and being indexed by BDC Search. The following search results display the profile pages that contain the search keyword (s).  Notice in Figure 8 how a search for Great Lakes will display the profile pages for both this customer and their orders.

Figure 8  BDC Search Results in Profile Pages

Enhancing the Profile Page

By default, the profile page is hosted on the Shared Services Provider site collection where the BDC administration site is located. The search result (that is, the URL) links back to the Shared Services Administration site collection address that is not scalable when you extend the Web application. In real world, we may not want to give end users access to the SSP Admin site for security reason. A better practice is to create our own profile pages and host them in any site that is not part of the SSP site collection.
In this example, we’ve created a dashboard page in Report Center to replace the default Profile Page. The Report Center provides enterprise reporting capabilities that are scalable and centralized.  We can direct users to custom profile pages by updating the View Profile action.

Figure 9  Replace the default Profile Page with Custom Report Center

As an alternative, we can also replace the default the URL in Action element in ADF with the path to the Profile.aspx page:

<Actions>
<Action Position=”1″ IsOpenedInNewWindow=”false” Url=”http://reportcenter/Pages/Profile.aspx?EmployeeID={0}” ImageUrl=”/_layouts/1033/images/viewprof.gif” Name=”View Profile”>
<ActionParameters>
<ActionParameter Index=”0″ Name=”EmployeeID” />
</ActionParameters>
</Action>
</Actions>

An action to call the Profile.aspx page for each Employee ID has been defined. This action will be the default action provides in the search result.

Now let’s create Profile page to display information about the Employee entity in Report Center. (The Report Center is a site that provides a central location for business-intelligence-related information.)

1. In the http://reportcenter site, create a new Blank Web Part Page called “Profile.aspx”.
2. Add the following two web parts: Query String (URL) Filter Web Part and Report Viewer Web Part.
3. Configure the Query String (URL) Filter Web Part to pickup parameter “EmployeeID” from the URL query string.
4. Configure the Report Viewer Web Part to display the employee profile report called “Employee Profile”.
5. Connect the Query String (URL) Filter Web Part parameter to the SQL Server Reporting Services Report Viewer Web Part parameter “EmployeeID”.

Figure 10  SQL Server Reporting Services Report Viewer

Note: Please make sure you use this Web Part and not the OOB Report Viewer Web Part.

Test that the page is picking up the URL string parameter “EmployeeID” correctly by typing the following URL: http://reportcenter/Pages/Profile.aspx?EmployeeID=108

Figure 11  Custom Profile Page in Report Center

Creating Custom Actions

Creating custom actions requires custom code. The approach we take is to treat the BDC source simply as a data reader, and to build our CRUD (create/read/update/delete) logic in custom code. The BDC object model provides the means to retrieve collections, list items and enumerators, and we can use GenericInvoker to perform CRUD, which has been extensively covered in Part 7. We get the Web Parts to display data, and then we can use custom actions to create and update records by using Web parts and InfoPath Forms.

Creating Web Parts

We have created and deployed two Web Parts using the Object Model as shown below, with one inserting a record and one updating an existing record. (Special thanks to Lightning Tools! I’ve used BDC Meta Man to help with creating the Web parts.)

Figure 12  The two Custom Web Parts can be used to add and update employee records

Add these two actions in the SSP Admin page:

Figure 13  Adding actions in the SSP Admin page

Now you can access the two actions from the content menu in the Employee List:

Figure 14  Custom Actions Add Employee and Update Employee in the context menu

Figure 15  Add Employee Web Part

Figure 16  Update Employee Web Part

Creating InfoPath Form

MSDN has a great article titled Using Business Data Catalog Actions to Pass Parameters to InfoPath 2007 Browser Forms that walks you through this in detail.  So I’ve created our InfoPath Form and added the action using the Form Template path:
http://dpm/_layouts/FormServer.aspx?XsnLocation=http://dpm/FormServerTemplates/EmployeeUpdate.xsn&DefaultItemOpen=1&EmployeeID={0}&FirstName={1}&LastName={2}

Figure 17  Adding Custom Action that uses InfoPath Form

Figure 18  Update Employee in InfoPath Form

Conclusion

In this final article, we have focused on actions that users can take on the business entities. We can define actions in ADF or add it “after the fact”. We have also created custom actions using the BDC object model by using custom Web parts and InfoPath forms to enable CRUD (create/read/update/delete) operation support. Once added, actions can be performed from within the list, which becomes relatively transparent to the user where the data comes from. [The statement that the BDC data is read-only is not true anymore as we have just demonstrated.] As you have learned from our 8-part series, BDC enables user interaction in business intelligence, enterprise content management, collaboration and social features, and enterprise search.

Looking Forward to SharePoint Server 2010

BDC has evolved to what is now called BCS (business connectivity services) in SharePoint 2010. Even though we can use BDC to enable connectivity to external systems, it was difficult to create solutions because of the lack of a designer and it was not so simple to create a solution that enabled users to make changes and write that data back to the external store. BCS now enables you to integrate LOB data into SharePoint and Office with full CRUD support, tighter integration with Office client applications, and better tools for modeling business entities. BCS is all about connecting to external data. We look forward to sharing with you the exciting new features in BCS and what they can do for you in the near future. Until then, aloha!

For your reference, these are our previous six articles:

1. Part 1, BDC Purpose and technical architecture
2. Part 2, Application Definition File (ADF) and its Development
3. Part 3, Developing an ADF to Connect to Web Services
4. Part 4, Consuming Business Data through Web Parts and SharePoint Lists
5. Part 5, Implementing Enterprise Search with Business Data
6. Part 6, Integrating User Profiles with Business Data

Until now, we have been working strictly with the out-of-the-box features and capabilities of BDC. MOSS Enterprise has also provided us another powerful approach for working with BDC, which is to employ the BDC Object Model and programmatically interact with a LOB system. This gives us as developers the power and control over BDC to further meet growing business needs.

You might be wondering why we use the BDC Object Model when you can just use ADO.NET. The reason is two-fold: 1) you already have BDC in place that has already abstracted the complexity of the business data and you want to maximize your investments, and 2) BDC takes care of authentication, authorization, and other security requirements, which is extremely helpful if you are writing a custom application that calls into a back-end system. So this makes the BDC Object Model a compelling API for us.

Using the BDC Object Model, we can accomplish many things such as the following examples:

1. Auto-populate a SharePoint list that has a Business Data Column
2. Directly edit a customer in SAP
3. Synchronize content between SharePoint and an LOB system
4. Update and manipulate the ADF schema

We now know what the Object Model can do so let’s first exam where the BDC Object Model fits in BDC architecturally.

The BDC Object Model Overview

To see where the BDC Object Model fits in the BDC world, let’s revisit the BDC architecture from Part 1 with Custom Applications highlighted in Figure 1.

Figure 1 The BDC Object Model Creates Custom Applications.

As you can see from the figure above, BDC allows developers to connect to disparate LOB systems and these systems are exposed as data sources to various MOSS features such as Web Parts, list columns, enterprise search, user profiles, and custom applications that can be developed to interface with SharePoint.

Custom applications can be also developed to customize and extend the existing capabilities of BDC. That’s where the BDC Object Model fits in the picture and becomes the focus of this article. The BDC Object Model provides us with new namespaces and classes (methods and properties) we can use in Visual Studio the same way we use a standard set of .NET classes and interfaces. The OOB Web Parts such as Business Data List Web Part goes through this same API that we are going to learn about. You can also create your custom Applications using the same API.

As we have learned already, BDC abstracts the business logic in an XML-based ADF file and stores it in a shared service provider (SSP) database. This file contains all the necessary configuration data that determines how the BDC will connect to the remote systems and which requested data to make available through the BDC Object Model. The following figure shows the interaction between the ADF and the Object Model.

Figure 2 The BDC Object Model and How It Works

As you can see, BDC Object Model consists of Runtime and Administration object models. Each of these performs different tasks:

1. Runtime object model

• Browse metadata, execute methods, retrieve instances, traverse relationships
• Used to build custom applications

2. Administration object model

• Create, read, update, and delete metadata and manage permissions
• Used to edit the metadata internally or to build metadata management tools

Using the BDC Object Model

In order to connect to the BDC from our code, we will use the root namespace within the BDC Object model Microsoft.Office.Server.ApplicationRegistry. If you’re curious as to the term Application Registry, just know that it was the original name for BDC. It’s defined in the assembly Microsoft.sharepoint.portal.dll. Through the root namespace we have access to the child namespaces and its classes that we are going to use later in the article. Here they are listed in this table:

Table 1: Namespaces and Classes

Tip: MetadataModel and Administration use some of the same class names such as LobSystem and Entity so one way to avoid conflict when using them together is to use an alias as follows:

using BDCAdmin = Microsoft.Office.Server.ApplicationRegistry.Administration;

MSDN already has a code sample linked here on how to use the Administration object model to create and import an ADF. Since it’s more practical to us, we will use the Runtime object model primarily throughout this article.

Now we have the basic understanding of the BDC Object Model. Let’s make use of it by applying it to the following three scenarios:

1. Auto-populate a SharePoint list that has a Business Data Column
2. Create a BDC Utility Class that can be used in custom Applications and Web services
3. Create a custom application to update the LOB systems

Since all three of these will use the Runtime object model, we can initialize our code examples by first referencing the assembly Microsoft.sharepoint.portal.dll and then import the namespaces accordingly depending on the tasks:

Now that namespaces have been imported, the next line of code designates the Shared Service Provider (SSP) that provides the ADF. This is accomplished using the following call to the SqlSessionProvider singleton Instance property:

using Microsoft.SharePoint;
using Microsoft.Office.Server.ApplicationRegistry.Runtime;
using Microsoft.Office.Server.ApplicationRegistry.Infrastructure;
using Microsoft.Office.Server.ApplicationRegistry.MetadataModel;

SqlSessionProvider.Instance().SetSharedResourceProviderToUse(sspName);

The next set of code is the actual connection to the appropriate BDC and system instance. The connection is established by indicating the LobSystemInstance and Entity objects with the following statements:

LobSystemInstance lobInstance = ApplicationRegistry.GetLobSystemInstanceByName(“SynergyData”);

//Connect to the customers entity

Entity customers = lobInstance.GetEntities()["Customers"];

Following are the top steps we perform to call into the BDC and retrieve the data:

1. Connect to the named SSP
2. Connect to LOB Instance
3. Connect to entity
4. Connect to method for the entity
5. Execute the method
6. Parse the results

Now that you understand how to initialize your projects that use the Runtime API, let’s now examine each scenario:

1. Auto-populate a SharePoint list with Business Data Column

This application automatically populates a SharePoint list that has a Business Data Column storing customer records. First we create a SharePoint teamsite named teamsite and a SharePoint list called Customers. Then we reference both the SharePoint assembly Microsoft.SharePoint.dll and BDC assembly Microsoft.SharePoint.Portal.dll. Then code the using clauses to import the namespaces and code against the BDC Object Model by connecting to BDC and execute the method instance which will return all customers.

As my colleague Randy Williams has already demoed in his blog, here’s the complete code:

//Get SPList handle

SPSite site = new SPSite(http://MOSS);
SPWeb web = site.OpenWeb(“teamsite”);
SPList customerList = web.Lists["Customers"];

//Connected named SSP using the following call to the SqlSessionProvider singleton instance property
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(“SSP”);

//Connect to the LOBSystemInstance
LobSystemInstance lobInstance = ApplicationRegistry.GetLobSystemInstanceByName(“SynergyData”);

//Connect to the customers entity
Entity customers = lobInstance.GetEntities()["Customers"];

//Connect to our finder method for customers
MethodInstance getCustomers = customers.GetFinderMethodInstance();

//Execute finder method
IEntityInstanceEnumerator results = (IEntityInstanceEnumerator ) customers.Execute(getCustomers, lobInstance);

//iterate through each row returned and add a new item
while (results.MoveNext())
{
SPListItem item = customerList.Items.Add();
item["Customers_ID"] = EntityInstanceIdEncoder.EncodeEntityInstanceId(new object[] { results.Current["CustomerID"].ToString() });
item.Update();
}

2. Create a BDC Utility Class

We’re going to create a custom library that uses the BDC Object Model to connect to an ADF.  We treat the BDC source simply as a data reader and to build our create/update/delete logic in an assembly. The BDC object model provides the means to retrieve collections, list items and enumerators, and we do the rest. In this class, we are going to create the following three static methods:

1. ExecuteFinder()
2. ExecuteSpecificFinder()
3. ExecuteGenericInvoker()

Let’s go through the code one by one. Here’s the first static method named ExecuteFinder that returns all of the entity records.

namespace SharePointMag
{
public class BDCUtil
{
public static DataSet ExecuteFinder(string SSPName, string LobSystemInstance, string Entity)
{
DataSet ds = new DataSet(Entity);

//Connect to named SSP
if (SSPName != null)
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(SSPName);

//Connect to LOB Instance
LobSystemInstance lobInstance = ApplicationRegistry.GetLobSystemInstanceByName(LobSystemInstance);

//Connect to customers entity
Entity entity = lobInstance.GetEntities()[Entity];

//Connect to Finder method for customers
MethodInstance finder = entity.GetFinderMethodInstance();

//Execute Finder method
DbEntityInstanceEnumerator records = (DbEntityInstanceEnumerator)entity.Execute(finder, lobInstance);

//Load matching entities into a DataTable
DataTable entitiesTable = new DataTable();
while (records.MoveNext())
{
//Load each entity and include the Action URL
DbEntityInstance record = (DbEntityInstance)records.Current;
DataTable entityTable = record.EntityAsDataTable;
entityTable.AcceptChanges();
entitiesTable.Merge(entityTable);
}

//DataTable ordersTable = customerOrders.Tables.Add(“Orders”);
ds.Tables.Add(entitiesTable);
return ds;
}
}
}

The following code calls ExecuteFinder of BDCUtil class to return the customers records:

public static DataSet GetEntity()
{
return BDCUtil. ExecuteFinder(“SSP”, “SynergyDataInstance”, “Customers”);
}

The following is a Web service example to call the BDCUtil class to return all the customer records:

[WebMethod]
public DataSet GetCustomers()
{
string SSPName=“SSPPS”;
string strLOBInstance=“SynergyDataInstance”;
string Entity=“Customers”;
DataSet ds = new DataSet(Entity);
ds = BDCUtil. ExecuteFinder(SSPName, strLOBInstance, Entity);
return ds;
}

Note: For simplicity, we explicitly pass in the name of LOB Instance and Entity in our examples. They could also be “discovered” if you desire. Please check out this MSDN article titled How to: Get Started with Using the Runtime Object Model.

The second method named ExecuteSpecificFinder returns a single row of record identified by its ID.

public static DataSet ExecuteSpecificFinder(string SSPName, string LobSystemInstance, string Entity, object[] parameters)
{
DataSet ds = new DataSet(Entity);

//Connect to named SSP
if (SSPName != null)
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(SSPName);

try
{
//Connect to LOB Instance
NamedLobSystemInstanceDictionary lobInstances = ApplicationRegistry.GetLobSystemInstances();
LobSystemInstance lobInstance = lobInstances[LobSystemInstance];
Entity entity = lobInstance.GetEntities()[Entity];

//Get the specific mathcing record
DbEntityInstance record = (DbEntityInstance)entity.FindSpecific(parameters, lobInstance);

//Loading matching entity into DataTable
DataTable entityTable = record.EntityAsDataTable;

//Return the results as a dataset

//DataSet ds = new DataSet(Entity);
ds.Tables.Add(entityTable);
return ds;
}
}

The following code calls the ExecuteSpecificFinder of BDCUtil class to return one row of employee record.

public static DataSet ExecuteSpecificFinder()
{
object[] parameters = new object[1];
parameters[0] = 1; //Employee ID;
return BDCUtil.ExecuteSpecificFinder(“SSP”, “SynergyDataInstance”, “Employees”, parameters);
}

This is the returned employee record that’s displayed in XML format:

<Employees>
<Employees>
<EmployeeID>1</EmployeeID>
<LastName>Davolio</LastName>
<FirstName>Nancy</FirstName>
<Title>Sales Representative</Title>
<TitleOfCourtesy>Ms.</TitleOfCourtesy>
<BirthDate>12/8/1948 12:00:00 AM</BirthDate>
<HireDate>5/1/1992 12:00:00 AM</HireDate>
<Address>507 – 20th Ave. E.Apt. 2A</Address>
<City>Seattle</City>
<Region>WA</Region>
<PostalCode>98122</PostalCode>
<Country>USA</Country>
<HomePhone>(206) 555-9857</HomePhone>
<Extension>5467</Extension>
<Photo>FRwvAAIAAAA…truncated…</Photo>
<Notes>Education includes a BA in psychology from Colorado State University in 1970. She also completed “The Art of the Cold Call.” Nancy is a member of Toastmasters International.</Notes>
<ReportsTo>2</ReportsTo>
<PhotoPath>http://portal.synergy.com/hr/employeephotos/ndavolio.jpg</PhotoPath>
<LoginAccount>synergy\ndavolio</LoginAccount>
</Employees>

Note: The <Photo> element has been truncated to save space here.

Now let’s move on to GenericInvoker by adding ExecuteGenericInvoker method in our BDCUtili class. The ExecuteGenericInvoker method calls the GenericInvoker method instance to write back to LOB systems.

public static void ExecuteGenericInvoker(string SSPName, string lobSystemInstance, string entityName, string methodInstance, object[] parameters)

{ //Connect to named SSP
if (SSPName != null)
SqlSessionProvider.Instance().SetSharedResourceProviderToUse(SSPName);

NamedLobSystemInstanceDictionary instances = ApplicationRegistry.GetLobSystemInstances();
LobSystemInstance instance = instances[lobSystemInstance];
Entity entity = instance.GetEntities()[entityName];
MethodInstance methInst = entity.GetMethodInstances()[methodInstance];
entity.Execute(methInst, instance, ref parameters);
}

In our third example, we look at GenericInvoker and then call the ExecuteGenericInvoker method we’ve just added.

3. Create a Custom Application to Update the LOB systems

BDC is very well known as read-only tool. It’s read-only because the OOB functionality does not make use of GenericInvoker methods. However, the object model and ADF support this type of method instance that we invoke just as we would invoke any other method. If you think of this as a stored procedure call and if I can call a stored procedure that runs a select statement, I should be able to call a stored procedure that inserts or updates as long as the underlining data repository supports them. In other words, the GenericInvoker method allows us to call other SQL commands. For our situation, we’ll use it to write back to our back-end system.

So let’s first add the GenericInvoker method in our ADF file and then call the ExecuteGenericInvoker method in our BDCUtil class . (Note: I’ve used BDC Meta Man to help generate the ADF and create ExecuteGenericInvoker. Thanks Lightning Tools! If you use BDC Definition Editor, you cannot select byte array byte[] as the return type of the field. You can instead select byte and edit the ADF by hand by changing byte to byte[].)  The following is the ADF and for the purpose of this example, only a single entity called Employees has been defined.

<Entity EstimatedInstanceCount=“0“ Name=“Employees“>
<Identifiers>
<Identifier TypeName=“System.Int32“ Name=“EmployeeID“ />
</Identifiers>
<Methods>
</Methods>
</Entity>
<Method Name=“insertEmployees“>
<Properties>
<Property Name=“RdbCommandText“ Type=“System.String“>
Insert into Employees(LastName,FirstName,Title) Values(@LastName,@FirstName,@Title);select NewEmployeeID = SCOPE_IDENTITY()</Property>
<Property Name=“RdbCommandType“ Type=“System.String“>Text</Property>
</Properties>
<Parameters>
<Parameter Direction=“In“ Name=“@LastName“>
<TypeDescriptor TypeName=“System.String“ Name=“LastName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@FirstName“>
<TypeDescriptor TypeName=“System.String“ Name=“FirstName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@Title“>
<TypeDescriptor TypeName=“System.String“ Name=“Title“ />
</Parameter>
<Parameter Direction=“Return“ Name=“Employees“>
<TypeDescriptor TypeName=“System.Int32“ Name=“NewEmployeeID“ />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name=“EmployeesInserter“ Type=“GenericInvoker“ ReturnParameterName=“Employees“ />
</MethodInstances>
</Method>
<Method Name=“updateEmployees“>
<Properties>
<Property Name=“RdbCommandText“ Type=“System.String“>Update Employees SET LastName=@LastName,FirstName=@FirstName,Title=@Title WHERE(EmployeeID=@EmployeeID)</Property>
<Property Name=“RdbCommandType“ Type=“System.String“>Text</Property>
</Properties>
<Parameters>
<Parameter Direction=“In“ Name=“@EmployeeID“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
<Parameter Direction=“In“ Name=“@LastName“>
<TypeDescriptor TypeName=“System.String“ Name=“LastName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@FirstName“>
<TypeDescriptor TypeName=“System.String“ Name=“FirstName“ />
</Parameter>
<Parameter Direction=“In“ Name=“@Title“>
<TypeDescriptor TypeName=“System.String“ Name=“Title“ />
</Parameter>
<Parameter Direction=“Return“ Name=“Employees“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name=“EmployeesUpdater“ Type=“GenericInvoker“ ReturnParameterName=“Employees“ />
</MethodInstances>

</Method>

<Method Name=“deleteEmployees“>
<Properties>
<Property Name=“RdbCommandText“ Type=“System.String“>Delete Employees WHERE(EmployeeID=@EmployeeID)</Property>
<Property Name=“RdbCommandType“ Type=“System.String“>Text</Property>
</Properties>
<Parameters>
<Parameter Direction=“In“ Name=“@EmployeeID“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
<Parameter Direction=“Return“ Name=“Employees“>
<TypeDescriptor TypeName=“System.Int32“ Name=“EmployeeID“ />
</Parameter>
</Parameters>
<MethodInstances>
<MethodInstance Name=“EmployeesDeleter“ Type=“GenericInvoker“ ReturnParameterName=“Employees“ />
</MethodInstances>
</Method>

I’ve created three GenericInvoker MethodInstances and they are EmployeesInserter,EmployeesUpdater, and EmployeesDeleter. When a method instance is executed, it will eventually call the appropriate SQL command as shown here:

• Insert into Employees(LastName,FirstName,Title) Values(@LastName,@FirstName,@Title);select NewEmployeeID = SCOPE_IDENTITY()
• Update Employees SET LastName=@LastName,FirstName=@FirstName,Title=@Title WHERE(EmployeeID=@EmployeeID)
• Delete Employees WHERE(EmployeeID=@EmployeeID)

The following method ExecuteGenericInvoker is to call the GenericInvoker method:

public static void InsertEmployee()

{
object[] parameters = new object[3];
parameters[0] = “He”; //Last Name;
parameters[1] = “Wen”; //First Name;
parameters[2] = “SharePoint Architect”; //Title;

try
{
BDCUtil.ExecuteGenericInvoker(“SSP”, “SynergyDataInstance”, “Employees”, “EmployeesInserter”, parameters);
}

catch (Exception exception)
{
Console.WriteLine(exception.ToString());
}
}

The result is as you can see here highlighted:

In the same token, the following code will update the employee record:

In the same token, the following code will update the employee record:

public static void UpdateEmployee()
{
object[] parameters = new object[4];
//The parameters must match the parameters defined in ADF
parameters[0] = “10″; //Employee ID;
parameters[1] = “He”; //Last Name;
parameters[2] = “Wen”; //First Name;
parameters[3] = “MOSS Architect”; //Title;

try
{
BDCUtil.ExecuteGenericInvoker(“SSP”, “SynergyDataInstance”, “Employees”, “EmployeesUpdater”, parameters);
}

catch (Exception exception)
{
Console.WriteLine(exception.ToString());
}
}

Result is as follows:

Note: the order of the parameters must match that of the parameters defined in the updateEmployees method in the ADF. The ReturnParameterName (ReturnParameterName=”Employees”) is required in the EmployeeUpdater and EmployeeDeleter MethodInstance, even though they have no parameters to return.

As we have just learned, the GenericInvoker method is extremely useful because it takes care of authentication and authorization while enabling our custom applications to call in and execute business logic stored in the back-end to insert/update/delete LOB systems.

Write-back Best Practice

Even though BDC can update the LOB systems, it’s not suited for all scenarios. I strongly recommend you to take it into consideration governance and change management that your organization might require. Here are some additional best practices:

1. Understand the vendor support implications for changes that you make. For example, directly querying SharePoint databases is not Microsoft supported because the risk involved in damaging it.
2. Understand data integrity rules before using GenericInvoker method to make direct changes.
3. Have error handling in place.
4. When full inserts and updates are needed, consider using the LOB vendor’s own API that provides transactional write-back capabilities from the client to the underlying LOB application. Another way this can be done is using custom actions which will be covered in detail in Part 8.

Conclusion

BDC Object Model provides a new and exciting solution by allowing direct integration with LOB systems. In this article we have explored the BDC Object Model and exploited the capabilities of the BDC from code. BDC encapsulates the business logic of the back-end systems and developers can just work with the BDC objects to interact with those systems. We’ve demonstrated how to retrieve customer records to populate a SharePoint list that has a BDC data column, and then created a BDC utility class to act as a data layer, finally called the utility class to update the back-end system. With this you can now develop custom applications to interface with SharePoint using the BDC Object Model.  And for organizations that have invested in BDC, you are on your way to leveraging the real power of BDC!

In case you’re just joining us in the series, here are our previous five articles:

Part 1.  BDC Purpose and technical architecture

Part 2.  Detailed overview of the Application Definition File (ADF) and how to develop one to connect to a back-end database

Part 3.  Developing an ADF to connect to Web Services

Part 4.  Consuming Business Data through Web Parts and SharePoint lists

Part 5.  Implementing Enterprise Search with Business Data

Overview

User Profiles within MOSS are a great way to store key information about SharePoint users.  This information, stored within individual profile properties, contains useful content such as first name, last name, date of hire, the user’s manager, and numerous personal details such as interests and skills.  This user metadata can be used when executing workflow, searching for people, utilizing social networking aspects such as my sites, defining audiences, any many other SharePoint capabilities.  SharePoint contains dozens of built-in properties, and you can add any custom properties as needed.

While all of this a great benefit to enhancing SharePoint’s knowledge of these people, there may be a drawback to this.  Here is yet another database of information that needs to be populated!  In most organizations some of this information lives within Active Directory (AD) and perhaps other details can be found in repositories such as HR systems, so you can say that SharePoint is really just duplicating what already exists elsewhere.  The primary goal of this article is to show you how you can eliminate this redundant data entry by pulling this authoritative information from your external systems through the BDC.

User Profiles and the profile properties are all maintained within your Shared Services Provider (SSP).  User profiles can be created manually, or you can create import connections which allows you to automatically synchronize these values from external systems.  Supported external systems are AD, LDAP (such as Lotus Notes), and BDC.

An important detail you must understand is that only AD or LDAP can be set up as a primary (or master) connection.  A BDC connection cannot be.  A primary connection means that a SharePoint profile with an account name (e.g. domain\username) can be created from this source.  BDC can only be a secondary connection, meaning that it is only able to fill in additional properties for an existing profile—in other words, it cannot create a profile.

To better explain, see Figure 1 below which shows the profile being created (based on the Account name) from AD and additional properties coming from an HR System (click image to enlarge):

Figure 1: Mapping profile properties between SharePoint and external systems

The arrows reflect the mapping between SharePoint profile properties and columns in external systems.  You can see that some information (such as first and last name) may be duplicated in external systems.  When defining mapping for a profile property, you can only specify a single source for the data.  Based on the arrows, the decision was made to pull the first and last name from AD.

For many organizations, this is a common scenario.  AD, maintained by the IT group, typically contains only a minimal amount of accurate user meta data.  Other systems, often maintained by the HR group, contains additional values.  SharePoint allows you to set up multiple primary and secondary connections to ensure all your profiles can be created and populated as needed.

Now that you have a basic understanding, the rest of the article will focus on how to create and configure these secondary BDC import connections.

Creating BDC Import Connections

Prior to creating a BDC Import Connection, I recommend you configure and ensure that your AD or LDAP primary connection is functioning.  Once this is working, you can then focus on your secondary BDC connection.  For my upcoming demos that I’ll walk you through, I am pulling accounts from a single domain inside AD.

To create a BDC Import connection, first access your User Profile and Properties screen within your SSP (Central Administration –> SSP –> User profiles and properties).  On this screen, click on the View Import Connections link.  Then click on the Create New Connection toolbar button.  Finally, choose Business Data Catalog type in the drop down list.  This will yield the following screen:

Figure 2: Creating a BDC Import Connection

In Figure 2 above, you’ll notice that you have two mapping options (depicted by the radio buttons),  1:1 and 1:many.  Let’s look at each option in detail:

1:1 Import Connection Type

A 1:1 connection is the most common type of connection that is used.  This means that a single profile in SharePoint will map to a single entity record that is returned from BDC.  For example, if we have a user profile for synergy\afuller as shown in Figure 1 above, this user will also exist in an external system.  For a 1:1 mapping, a Specific Finder method will be called based on the selected entity.  This shouldn’t surprise you as a Specific Finder method returns a single record from the external system.  A good question you might be thinking is how does it know which record to pull?  I’ll answer this question as I show you how to configure the connection.

To configure the 1:1 mapping as shown in Figure 2, we need to first start off by creating or identifying an existing profile property that can be used to uniquely identify the user in the external system.  This property must also match the identifier that is defined for your entity in your ADF.  In my example, I will be using EmployeeID which is the primary key for an Employees table in a SQL Server database.  Employees represents a table usually found in HR or ERP systems.  Here is the Identifier as reflected in the ADF:

Figure 3: EmployeeID as the Identifier for Employees Entity

And here is the Specific Finder:

Figure 4: Specific Finder for Employees Entity

As you can see, I have selected EmployeeID to be the unique identifier.  Since I don’t have a built-in profile property in SharePoint that stores EmployeeID, I’ll create one.  This is necessary for the 1:1 mapping to work.  Here is how it looks after I’ve created it:

Figure 5: Newly Created EmployeeID Profile Property

I chose the integer type because the property’s type must be compatible with the type defined in the Identifier (which was defined as System.Int32 as shown in Figure 3).

Now that this is done, I can create my BDC Import connection.  (Note: Even though you are only creating a 1:1 mapping that will use a Specific Finder, this screen also expects a Finder method.  Even though the Finder method is only used when creating 1:Many connections as discussed below, you must still have one or you will get an error when you select the entity from the catalog in this screen.)

Going back from Figure 2 above, after selecting Business Data Catalog as the type, you must enter a connection name.  I recommend a friendly name that you’ll easily recognize. For my demo, since I’m pulling employee information from an HR system, I’ll call the connection HREmployee.  Here is how it looks just prior to saving:

Figure 6: Creating a 1:1 Import Connection

You’ll notice that I selected the EmployeeID profile property that was just created.  For this to work, I now need to go into my current SharePoint profiles and manually enter in a values for EmployeeID.  This is necessary in my example since SharePoint must know what value to pass to the Specific Finder.  If you’re using a built-in profile property that is already populated (such as the Account name), you don’t need to do this.

Now that the 1:1 relationship between the User Profile and the external system is set up, it’s time to figure out which profile properties you want to import.  If you refer back to Figure 1 above, you’ll see that I illustrate the importing of Title and About me properties from the external system.

For the demo here, I’ll map two existing properties, Hire Date and About me.  You do this by editing the profile property (SSP –> User profile and properties –> View profile properties –> Choose a property and edit).  There’s no reason why I can’t create new properties and map these; the process is the same.  Here is how to map the Hire date profile property to the HireDate column from the HREmployees import connection:

Figure 7: Mapping Hire Date

Figure 7 above shows the upper and lower portion of the same screen.  When choosing a column in the Data source field to map drop down list, you’ll be presented with a list of all the columns returned from the Specific Finder.  So, the rule of thumb is the ensure that you are returning all columns you want to map from your Specific Finder.  As with creating the import connection, when mapping properties this way, you must ensure the data types are compatible.  If they are not, you’ll be given an error message like this: Cannot map to this data source field. This property is of type ‘date’ and the data source field you are mapping to is of type ‘System.String’.

I have also mapped the About me property to the Notes column defined in the Specific Finder.  At this point, the mapping configuration is in place.  We’re not quite done yet, however.

Another important step is to ensure that your Default Content Access Account has permissions to update these profile properties through the BDC.  The Default Content Access Account is configured when setting up Enterprise Search (SSP –> Search settings –> Default content access account).  If you prefer, you can also set a specific account when configuring the User Profile Import.  Just go to SSP –> User profile and properties –> Default access account.  Now that you know what your access account is, you must grant this account two sets of permissions:  1) Permissions to update your profile properties and 2) permissions to call into these BDC methods.

To grant permissions to update profile properties, go to SSP –> Personalization services permissions.  On this screen, grant your access account the Manage User Profiles permission as shown here:

Figure 8: Granting Content Access Account Manage User Profiles Permissions

If you’ve worked much with profile properties, you’ve probably noticed that each property can be individually secured by adjusting its privacy setting.  The Manage User Profiles permission grants my access account (SYNERGY\MOSS.Search) write permissions to all profile properties.  Without this, you’ll need to set the privacy level of each mapped property to Everyone, which may not be the best security level.

The second permission is to grant this access account permissions to the BDC entity.  If you’ve been following our series, we covered BDC permissions in Part 2.  The minimum permission you need to grant is Execute as shown below.  You access this screen from SSP –> View Applications –> <Select ADF> –> <Select entity> –> Manage Permissions.  In this screen shot, I grant MOSS.Search execute permissions to the Employees entity:

Figure 9: Granting Access Account Entity Permissions in BDC

Finally, we are now done and ready to import.  Let’s quickly recap to make sure we have these steps down.  Again, we’re talking about a 1:1 mapping.

1.  Ensure ADF is prepared with a Specific Finder returning the columns you want to map.

2.  Create a new or choose an existing profile property that contains values to match the entity identifier.  In my example above, I used EmployeeID.

3.  Create a 1:1 import connection.

4.  Map your desired profile properties to columns returned from Specific Finder.

5.  Grant your content access account permissions

To start the import, return to the main User Profiles screen and click Start full import.  This will first do a full import from your primary import connection.  The status here is reflected next to Profile import status shown below.  Once that is done (and sometimes with a brief minute or two delay after it finishes), the secondary imports from BDC will begin.  This is reflected next to Membership & BDC import status.  Both are shown highlighted here:

Figure 10: User Profile import status

Once the profile imports complete (and it may take from several minutes to possibly an hour for thousands of users), you’ll want to check the import log to get a sense on whether it succeeded.  To do so, click the View import log link.  When reading the log, you should know that a PEOPLE_IMPORT content source refers to individual person crawls through your primary import connection, whereas PEOPLE_DL_IMPORT refers to group (i.e. distribution list) crawls.  PEOPLE_DL_IMPORT is also where your secondary import connections through BDC are logged, so it’s relevant to us in the scope of this article.  Here is a part of the log showing both content sources for one user, Andrew Fuller.

Figure 11: User Profile Import Log

Even though the logs may look fine, you’ll certainly want to verify some of the mapped values come over.  To view one, simply click on View user profiles and select a user.  In Figure 11 below, here are the two mapped properties for synergy\afuller:

Figure 12: About me and Hire date values pulled in by profile import

Since I mapped the About me property, you can also see this within the user’s My Profile found in their My site.  Here is Andrew Fuller’s:

Figure 13: Andrew Fuller’s My Profile page

Now that you have a solid overview of 1:1 mappings, let’s turn our attention to 1:many.

1:Many Import Connection Type

While less common than 1:1 connections, a 1:many is handy in a few situations.  Let me give you one example.  As mentioned in the overview section above, the IT group typically manages basic user attributes in AD.  In all my years of consulting, I’ve encountered very few organizations that actually keep these attributes up to date.  The problem is that IT is usually busy handing a myriad of critical problems, and they don’t usually bother updating less important attributes such as someone’s manager.  As long as the permissions are right, that’s what is important to them.

This can cause problems, however.  One of the useful SharePoint profile properties is Manager which gives SharePoint knowledge of the organizational hierarchy.  This was shown in Figure 13 above when looking at Andrew Fuller’s My Profile page.  By default, this property is mapped to the manager setting for the user in AD.  However, if this value is not set or out of date, SharePoint won’t have an accurate picture of the organization.

To correct this problem, we will pull the manager from a Departments table that is stored in our external HR system.  Here is where the 1:many comes in.  Recognize that many users will have the same department.  The idea is that we want to use the department profile property in SharePoint, which is correctly set, and use that to look up into the Departments table (in the external system) to pull the manager for that department.  So, you can think of the 1:many to be for each department there will be many user profiles with that value.

Let me show you how I configure the mapping to pull the Manager property from our HR System.  I start off by adding another Import connection.  Yes, you can have multiple import connections coming from BDC.  I call this one HREmployeesGetManager and here is how it is configured:

Figure 14: Creating a 1:many import connection

When choosing the 1:many mapping radio button, you have two drop downs to fill in.  You first need to select the filter.  The choices you have will be the filter descriptors that have been made available for the entity’s Finder method.  Hence, to use 1:many mapping, you must have a finder method and at least one filter descriptor.  (Note: if you cannot select 1:many mapping, it is probably because you don’t have filter descriptors defined in your ADF.)

For my demo, here is the Finder method and filter descriptor for the Departments entity:

Figure 15: Finder method for Departments entity

In the SQL select query, you can see that I am selecting the DepartmentID, DepartmentName and Manager columns based on a certain department name.  The next step is to change the default mapping for the Managers property.  Here is how I have changed it:

Figure 16: New mapping for Managers profile property

For the Data source field to map, I have pointed this to the Manager column.  Other column choices are the other columns returned from the Finder method.  For the last text box (Manager field string format in AD), you can leave this unchanged.

That’s it.  Run another import, and you should now have your manager property coming from your external HR system.  For this to work, you will need to ensure your Department property is populated for your user profiles.  In my case, this value was still coming from AD.  That’s right, you can use a value that is imported from AD and pass that as a parameter to your BDC connection.  Pretty cool.

Let me make one last point on 1:Many import connections:  Be careful if your Finder method returns more than one record, and your mapped property does not allow multi-values.  For example, perhaps the HR system had two department records with the same name.  And for each record, a different manager is set.  When the import runs and it tries to pull the manager based on the name, two records will be returned. When this happens, the import process will take the first record it finds, and this may not be the one you want.  If you are using a multi-valued property, however, it works nicely and will store the values for all records that are found.

Troubleshooting Tips

When troubleshooting import problems, you’ll definitely want to go to the SharePoint logs (by default C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\LOGS).

When mapping, a profile property can be mapped to only one source (primary or secondary).  Similarly, a column from the Specific Finder (1:1 connection) or Finder (1:many connection), can only be mapped once.  For example, if you map the HireDate column to one profile property, you cannot also map it to another.

Remember that you can only map compatible data types.  Fortunately, you will get this error when you try to save your profile property.

When mapping strings, be careful about the column length.  If your profile property only supports 25 characters, and you try to store 50 characters that is the returned value from BDC, you will get an import error for that profile. You would only know this is a problem by seeing this error in the SharePoint logs.  (Note: you cannot change the length of any of the out-of-the-box profile properties.  In this case, you would need to create a new property.)

When creating BDC import connections using the UI, the entity selected from the catalog must expose both Finder and Specific Finder methods.  This is necessary, even if you plan on using only 1:1 or 1:Many connection types.

Conclusion

In this article, we covered everything you need to know about importing User Profiles from BDC.  And you now know as much about this as I do.

Coming up in Article 7, we will cover the BDC API.  This will allow you to call into the BDC programmatically from your custom applications, web parts, or any other .NET code you need to write.

If you’ve been following our BDC series, we’ve previously gone through the following topics:

1. Part 1, BDC Technical Architecture
2. Part 2, Application Definition File (ADF) and its Development
3. Part 3, Developing an ADF to Connect to Web Services
4. Part 4, Using BDC Web Parts to Consume BDC Data and Embedding BDC Data in Lists

• How can I find the customer information for Great Lake Good Market in CRM?
• How can I find the projects and resources details in Project Server 2007?
• How can I find an employee’s legal name in HR system?

With the help of the BDC Search, these all become possible. The Enterprise Search in MOSS can gather, index, and offer full-text search on all business data (entity instances) from applications registered in the BDC. Not only does it enable us to find answers in MOSS, but also it allows us to do so without having to leave SharePoint to go to these systems individually. It makes integrating search into existing line of business (LOB) applications like a CRM solution easier.

For example, if we register the Northwind sample database in the BDC, Enterprise Search can crawl it and return customer data in search results. In addition, the Enterprise Search feature’s extensibility enables us to create customized search experiences. We can create a custom search tab in the Search Center where our users can search for customers in the database, and we can also create a custom UI for the search results based on which we can also take action. So let’s dive right into it.

BDC Search Overview

As we’ve covered in prior articles, surfacing the BDC data to SharePoint using BDC Web Parts and lists is just first part of the BDC saga, it can also enable you to index and search on the BDC data. The following figure is an example of searching for information about a customer and then displaying both structured data from the database and unstructured documents and content stored within SharePoint.

Figure 1: Enterprise Search with BDC

Here’s another example of leveraging BDC to index Project Server data and enable us to search the different objects (projects, tasks, resources, custom fields, lookup tables, and so on). This is of great value because out of the box the Project Server data is not indexed. Only the objects in project workspace are indexed such as Issues, Risks, Documents, etc. For instance, if we are looking for project detail information for the Enterprise resource planning (ERP) project, we can search our Project Server and find the answers as shown in Figure 2:

Figure 2: Search Project Server with BDC

The BDC is truly a catalog of business applications that helps bridge the gap between MOSS and business applications by bringing in key data from various applications to SharePoint sites, lists, searches, and user profiles. Let’s revisit the BDC architecture from Part 1 with both XML Metadata and Search components highlighted in Figure 3.

Figure 3: BDC architecture with emphasis on the Search

Firstly, we’ve highlighted the XML Metadata because it is the key element in the Application Definition that makes the BDC Search possible. BDC operates off of an XML-based Application Definition File (ADF). The metadata in ADF enables MOSS to crawl the BDC data source and add them to the search index.

Secondly, we highlight the Search component because the search engine uses a protocol handler to crawl and a security trimmer to filter results. BDC uses these just like third party implementations when supporting BDC searching.

BDC Search enables us to quickly find content in business data through search queries. It also integrates with search scopes and search center to become an integral part of MOSS Enterprise Search. Here I’ve created a custom search tab and a search scope for the BDC data to create a unique search experience in which the BDC search results are segregated from the rest of SharePoint search results. See Figure 4.

Figure 4: Customized BDC Search Experience in Search Center

By enabling the search of BDC data, BDC Search extends the SharePoint search capabilities. This extends the reach of the back-end LOB systems. Today most LOB systems don’t have Full-Text search capability and BDC Search can serve for that purpose.

However, out of the box the BDC Search does not support indexing unstructured data like documents in a document management system. This is because the protocol handler for BDC is meant to connect to external structured data only. With custom protocol handlers, it’s possible to make MOSS index unstructured data . For example, using Web services to directly support BDC we can enable SharePoint to search, list, and retrieve documents from an external document management system via Web services.

Now let’s look into how to enable BDC Search.

Enabling BDC Search

One might think once we import an ADF, we can start to search. BDC Search doesn’t come automatically, and we must enable it. To enable BDC Search, we must first make sure these two methods exist in the ADF for the entity you want to crawl:

1. SpecificFinder
2. IdEnumerator

These are the only required metadata for Enterprise Search to use BDC to index the business application’s data.

In Part 2, we’ve covered the ADF and its vocabulary: System (data source), Entity (table), Method (operation on entity), and Association (relationship). There are three primary types of methods: Finder, SpecificFinder and IdEnumerator. An IdEnumerator method returns a searchable list of unique IDs for an entity. This list enables indexing of the entities and SpecificFinder is then used to find details for the entity instance based on the specific ID. Therefore, IdEnumerator and SpecificFinder work together to enable Enterprise Search to use BDC to crawl the business data.

Now since we have a good understanding of what it takes, let’s put it in action. We’re going to use Business Data Catalog Definition Editor, a free tool from the MOSS SDK, to create our ADF (shown in Figure 5 below). Though one of the limitations of this tool is that it doesn’t create a Finder method, BDC Search doesn’t need use this anyway, so it works very well for creating an ADF that’s just used for searching. You can learn how to use this tool here.

Incidentally, both MOSS BDC Design Studio and BDCMetaMan will create these methods necessary to implement BDC searching.

We begin by adding a LOB system for our sample database, a modified version of Northwind database. We’re going to use the Customers entity that resembles one in a CRM system (or the like).

Figure 5: Create Application Definition

To index and search for Customers entity in the database, we’re performing the following tasks:

1. Define an IdEnumerator method in the BDC definition (Figure 6)
2. Define a SpecificFinder method in the BDC definition (Figure 7)

Figure 6: Define a special type of MethodInstance method IdEnumerator.

The IdEnumerator method returns a list of CustomerID records, which will be indexed by MOSS Search. So now we know that the sole purpose of IdEnumerator here is to enable Search. Here’s the SQL query:

Figure 7: Define a SpecificFinder

In Part 1, I covered the purpose and technical architecture of the BDC.  A quick look at the BDC in action was also introduced.

In Part 2, I covered the Application Definition File (ADF) in depth and how to develop one to connect to a back-end database.

In Part 3, I continued the discussion of the ADF, but looked at how to develop an ADF to connect to Web Services.

In this Part 4, which is much less technical, the focus is two primary ways to consume BDC data through the UI.  This will include Business Data Web Parts and incorporating Business Data into SharePoint lists.

Business Data Web Parts

Business Data Web Parts are the most common way to surface BDC data to end-users.  Using them, users can filter and sort the data to quickly find the content needed.  Their flexibility and ease of use empowers users to perform many tasks that would otherwise require custom development.  In MOSS Enterprise edition, there are five Web Parts and we will cover each in this article:  Business Data List, Business Data Item, Business Data Related List, Business Data Actions, and Business Data Item Builder.  Each of these Web Parts will only provide a read-only view of the external data, so don’t plan on making any changes through them.  There are ways of updating the external data, but we will wait until Part 8 to get into this.

Business Data List Overview

The Business Data List is the most powerful and most commonly used Business Data Web Part.  At its heart, it is a Data View Web Part and is used to display a sheet of columns and rows (or items).  The items displayed are based on filter criteria that the user enters.  Here is a simple example where we find all customers located in the WA (Washington) Region.

Figure 1: Business Data List Web Part

These customers that you see are defined in a SQL Database, and the information is pulled in real time.  This is a powerful way to expose your line-of-business data within a centralized SharePoint interface.

Let’s quickly walk through how I got here.  As with any Web Part, you start off by editing a Web Page (Figure 2) and adding a Web Part into one of the Web Part zones (Figure 3).

Figure 2: Editing the Page from the Site Actions Menu

Figure 3: Adding a Web Part to a Zone

After clicking Add a Web Part, you will be presented with all the Web Parts that are available for that site collection.  In the Business Data category, you should see the five Business Data Web Parts.

Figure 4: Business Data Web Parts

After you select the Business Data List Web Part and click the Add button, you will be presented with this next screen, prompting you to now configure the Web Part.

Figure 5: Default display after adding a Business Data List Web Part to a zone

After you click the Open the tool pane link, you are presented with all the configuration properties for the Business Data List Web Part.  The required setting is that you must select the Type from the catalog as shown here.

Figure 6: Specifying the type

The Type refers to the entity in one of the ADFs that have been loaded.  The entity must also have a finder method defined.  You can either enter in the name directly and click the Check Types icon .  Or, you can select the Browse icon to see what is available.  (Security Note: Only those applications and types that you have Select in Clients and Execute permission on will be shown.)  In the image shown below, one of the four types can be selected.

Figure 7: Browsing applications and types stored in the catalog

In the first screen shot above (Figure 1), you saw a list of customers.  To get this, I selected the Customers type from SynergyData (the name of the BDC Application).  Initially, this will return all of the columns that are defined in the Finder method within the ADF as shown below.  (Note: I also exited Edit Mode to return to the regular view).

Figure 8: Business Data List Ready for Action

At this point, the Web Part is ready for filtering and sorting operations.  To filter, you first choose one of the Filter choices from the drop down list shown on the top left as shown here:

Figure 9

What is shown are the filter options that are defined by the FilterDescriptors as covered in Part 2.  Here is what is defined for this Customers entity in the ADF:

Figure 10

If the type of filter is a Wildcard, you will have several filter options that you can select in the second drop down list as shown here:

Figure 11

If you’ve specified Comparison, you only have one choice:

Figure 12

You can add additional criteria by clicking the Add button and adding another filter.  These are all processed as an “And” operation, meaning that only items that match all criteria are shown.  To sort, as with any SharePoint view, you simply click on any of the columns headings.

You can also filter by using the column headings.  In this example, I start off by searching for all customers in the SP region.  This initially returns six items.  I then filtered the display to only show those in Sao Paulo.  The advantage is that you can still perform some filter operations even though the column is not defined as a filter descriptor.  Here is the result:

Figure 13: Column Filtering on City

What if you wanted to just filter columns?  For example, perhaps you only need to show a handful of columns for each customer.  This is easy and is done by editing the default view for the Web Part.  To do this, you can go back into edit mode for the page and click the Edit View link as shown here:

Figure 14: Editing Business Data List View

This will then take you to a screen where you can adjust the view in a number of ways.  One of the choices is to select the columns as their position.  As you can see below, this works just like a regular view for a SharePoint list.

Figure 15: Selecting columns to display

From this same screen, you also have some other useful features.  For example, you can “hard code” a specific filter so that only certain items display.  For example, perhaps each region has its own SharePoint page, and on that page, you only want to display the customers in that region.  You could configure the settings as shown here and prevent the user from changing the filter:

Figure 16: Only show customers in the SP Region

Here is the result:

Figure 17: Customers in the SP Region (and user cannot change the filter)

While editing the view, you have many more options such as the default sort and additional filter options.  I’ll leave these to your own study.

Business Data List Advanced Settings

Let’s turn our attention to more advanced capabilities.  Since the Business Data List Web Part is a type of Data View Web Part, you have a number of settings such as customizing the XSL or Parameters.

The XSL (eXtensible Stylesheet Language) is an XML-based transformation language that converts XML that is returned from BDC into HTML that is displayed for the Web Part.  This XSL is generated automatically when you select a catalog type such as customers.  While you can edit this XSL directly inside a dialog box for the Web Part, it is much easier to copy and paste this into a file and edit within Visual Studio, SharePoint Designer or your favorite XSL editor.  Keep in mind that the XSL contains not only the HTML results, it also contains the HTML for the selection criteria, the Actions toolbar, and the column headers.

A simple XSL change would be to enable alternating line styles.  The result looks like this:

Figure 18: Modified XSL to highlight alternating items

To enable this, I simply inserted the <xsl:if> condition you see here highlighted in this code snippet:

Figure 19: XSL

When defining the XSL, you can either link to an XSL stylesheet (located within a document library) or define the XSL for this specific Web Part instance.  Linking the XSL would be best for identical Web Parts that you have in different locations.  Since the XSL is generated for the specific columns, a shared XSL file cannot be used different Business Data Lists with different types (entities).  Having a linked XSL sheet is also helpful to ensure that changes made to the Web Part will not erase your custom XSL code.

For those not too comfortable with XSL, you have some control over the Web Part using SharePoint Designer (SPD).  Like the changes to the view we saw above, SPD also allows you to define filtering and sorting behavior.  One advantage with it, however, is that you can define filters based on variables such as the query string.

To help explain, let’s adapt the Region example we just covered in Figure 17 above where we hard coded the filter for the SP Region.  In this case, let’s pull that Region from the query string instead which would give us a more flexible design.

This section assumes you have a basic understanding of editing pages in SPD.  For a brief overview, see my friend Ishai Sagi’s introduction here.  Once in SPD, open up the page for the web site where the Web Part lives.  We click on the Web Part and access the Common Data View Tasks by clicking on the tiny icon as highlighted here in red:

Figure 20: Customizing Web Part in SPD

Once you click the icon, it will expand the Common Data View Tasks dialog as shown here:

Figure 21: Common Data View Tasks Dialog

From this point, click the Parameters link.  Click the New Parameter button and fill in the contents as shown below.  This will create a new parameter named Region that will be pulled from the query string.

Figure 22: Adding a New Parameter

Now that the Region parameter is created, we can create the filter that is based on its value.  Return to the main Common Data View Tasks dialog and this time select Filter.  Create a new filter by as you see here.  Note: The [Region] value is the parameter that you just created.

Figure 23: Adding a New Filter Based on the Parameter

Save all of your work in SPD and refresh the page in the browser.  If you do nothing else, you’ll find that your Business Data View Web Part does not return any items.  This is because the Web Part is expecting a Region query string value.  Edit the URL in the address bar, and add the query string and value after your page.  For example:

http://WebApp/teamsite/default.aspx?Region=SP

Now run a new query using the Web Part.  You should find that all the item results are automatically filtered to only the SP region.  If you prefer, you can supply a default value for the parameter when you create it as shown in Figure 22 above.

Important Note: When editing the Web Part using SPD, it will change the current XSL and Parameter settings.  So, be careful if you will be customizing the XSL manually and also through SPD as it will overwrite your changes.

Business Data Item Web Part

Now that we have thoroughly covered the Business Data List Web Part, the remaining Web Parts are much easier to learn.  The next one on our list is the Business Data Item.  This Web Part is intended to just show you the column values for a single item that is returned from BDC.  It is often used in conjunction when you connect Web Parts as you’ll see shortly.  Before we go there, let’s first look at a simple example.

Here you see the contents of one customer:

Figure 24: Business Data Item Web Part

To configure the Business Data Item Web Part, first set the Type from the catalog picker.  Then specify the item by searching for it.  Here is how it the configuration looks:

Figure 25: Configuring the Business Data Item

While this is easy to do, it’s not too practical to hard code in a single item.  Instead, let’s see how you can connect this to our Business Data List Web Part.  This will allow you to select a customer from the list and then display the customer’s details.  To start, add a Business Data List and Business Data Item Web Part on the same page.  They can be in different zones but must be on the same page.  You must also make sure they are both set to the same Type (such as Customers).  From here, you need to establish a Web Part connection between them. You can do it as shown here:

Figure 26: Connecting Business Data List to Business Data Item

This will send the selected item from the Customers List (on the left) to the Customers Item (on the right).  Once these are connected, you get the display as shown below.  Notice that selecting The Cracker Box shows this customer in the Business Data Item Web Part on the right.

Figure 27: Result of Web Part Connection

Business Data Related List Web Part

The Business Data Related List Web Part is used when you want to pull related items using an association defined in the ADF.  For example, perhaps we want pull the sick days a certain employee has taken.  Or we want to list the orders for a selected customer.  Let’s see how this latter example would work.  To start, let’s add a Business Data List and Business Data Related List Web Part to our page.  As we have been doing, we configure the Business Data List to use the Customer Type in the catalog.  We then point the Business Data Related List to the Orders Type.  Here’s how the Business Data Related List looks when configuring it:

Figure 28: Configuring the Business Data Related List

Notice that you must specify the relationship on the Orders Type that you want to use.  This choice is provided to us since it was defined in the ADF as you can see here:

Figure 29: Association between Customers and Orders in the ADF

Now that the Web Parts are configured, we need to establish a connection between them.  Once that is done, we get the following result:

Figure 30: Result of Connecting Business Data List and Business Data Related List

Of course, if you prefer, you can have the selected customer drive both the Orders List and the single customer in the Business Data Item.  Here’s how it would then look:

Figure 31: Connecting Three Web Parts Together

Business Data Actions

Actions are a common way of displaying or editing a single item, and we will cover them in detail within Part 8.  To give you a basic overview, let’s look at a sample action that is defined in the ADF.  Here is one that is defined for our Customers entity:

Figure 32: Action Defined for Customer Entity in ADF

Notice that the action has a name (in the figure above it is View Profile) and contains one or more ActionParameter elements.  This column (or columns) is often the Identifier for the entity.  This means it is used to uniquely identify this item.  For our customer’s entity, the CustomerID will be provided as a query string variable to a page called dbo.Customers.aspx.  How this is processed is covered in the Business Data Item Builder Web Part a bit later.

Now that you understand the basics of actions, the Business Data Actions Web Part simply lists all of the actions defined for a single item.  Here is how the configuration for the Web Part looks:

Figure 33: Configuring the Business Data Actions Web Part

Notice that you configure it much like the Business Data Item Web Part.  The only difference is you can specify which actions to display and how should they be displayed (i.e. the Style).  The Style options are Bulleted List, List, and Tool bar.  Once you have it configured (and connected to a Business Data List Web Part), here is how the results look with a Bulleted List:

Figure 34: Connecting the Business Data List and Business Data Actions

When you click on the View Profile link, it takes you to the Url defined for this action which is http://spserver:22222/ssp/admin/Content/dbo.Customers.aspx?CustomerID=LAZYK.  Incidentally, you can also access these actions by clicking the View Profile context menu as shown here:

Figure 35: Accessing the Actions from the Business Data List

As you can see, this is a flexible way of sending a single item to any web page, whether this is in SharePoint or an external web site.  We’ll look at other possibilities with Actions in Part 8.

Business Data Item Builder

The Business Data Item Builder is a special, invisible Web Part that is responsible for extracting the value from a query string variable and delivering it to a Business Data Item Web Part.  By invisible, I mean it is only visible in edit mode; when in normal operation, it is hidden.

When adding this Web Part to a page, there is no configuration on it other than connecting it to a Business Data Item Web Part.  The way it works is by extracting a variable from the query string (for example CustomerID) and sending it to the Business Data Item Web Part.  It knows what variable to extract by reading the ActionParameter defined in the ADF.

A good way of understanding how the Business Data Item Builder Web Part works is by looking at one of the profile pages.  A profile page, which I will cover in Part 8, is a page that typically displays a single item in some way.  Profile pages can be created automatically when you import the ADF.  For an automatically created profile page, this is comprised of a Business Data Item Builder and a Business Data Item Web Part.

To demonstrate, when clicking the View Profile link shown in Figure 35 above takes us to the auto-generated profile page as shown below.  Notice you only see one Business Data Item Web Part.

Figure 36: Profile for a Single Customer

If we edit the page, however, you’ll notice the Business Data Item Builder.  This web part is automatically connected to the Business Data Item Web Part.

Figure 37: Editing the Profile Page for Customers

That is the only purpose for the Business Data Item Builder, so you will probably not find yourself using it all that often.

With that, this wraps up our discussion on the out of the box Web Parts for the Business Data Catalog.  Let’s turn our attention to the second (and much shorter) half of this article.

Incorporating Business Data within Lists

While the Business Data Web Parts are useful, sometimes you just need to embed external data right within a SharePoint list or library.  For example, perhaps you already have a contact list in SharePoint for all your customers, and you are maintaining this separately from your CRM system.  A better solution would be to add Business Data columns to your contact list and pull in some of this data from CRM.  This means you would only maintain the customer entry in CRM, and these changes are pulled into SharePoint.

Another advantage is that you can see information from multiple sources all merged together.  Let’s say that a list of the products your company sells is stored within a SharePoint picture library.  You can add a Business Data column to the library which can pull the current product name and inventory level from an external ERP database.  Now you can see the product’s picture, the product name, and its inventory in one consolidated view.

When you add a Business Data column to a list, SharePoint will pull the data from the external system and store a cached copy of it in the list.  This is quite different from Web Parts where you are always pulling the real-time value from the external system.  The advantages of caching the data in the list are faster performance and the flexibility of having the data stored right in SharePoint.  The drawbacks are the that the additional columns of data will increase the size of the list (and content database).  Also, for volatile data that changes often, the cached data will quickly get stale.  As you might guess, the common-sense rule of thumb is to only pull data that does not need real-time status.  However, as you will see, it is easy to refresh the cache.

Just like the Business Data Web Parts, you should consider the cached data in the list as read only, and SharePoint does not give you an interface to change it.

Now that you have a basic idea how this works, let’s take a look at the picture library example that pulls the current product name and inventory value from a database.  For this demonstration, I already have a Products entity defined within an ADF.  I have created a picture library that has three product pictures added.  The All Pictures view looks like this:

Figure 38: Initial View of the Pictures Library

From the Settings menu, let’s add a Business Data column. The column named will be called Product Name as you can see here:

Figure 39: Adding a Business Data column called Product Name

When you add a Business Data column, you need to specify the Type, just as you would with the Web Parts.  Below you can see I have selected the Products Type.  I have also added two columns as you can see here

Figure 40: Selecting the Columns from the Products Type

Notice that you can add as other columns if you like.  I have added the ProductName (the main column in the drop down list) and UnitsInStock which gives us the current inventory value.  After we add these two columns, here is how the All Pictures view now looks:

Figure 41: All Pictures View after Adding Business Data Column

This is nice, but you can see that it doesn’t automatically pick up the values from the database.  When you add a Business Data column to a list, you need to map up each item in the list to the matching item in the database.  This is a 1:1 mapping.  Here is how you could manually do this using the UI.  Go to the edit properties for an item, and you’ll see the Business Data column as you see here:

Figure 42: Matching up Product Name Item

In the text box, you can type in a unique product name and click the Check Names icon .  Or you can also click the Browse icon and search for it.  Here is how the browse works:

Figure 43: Finding Matching Row

In the figure above, we are editing the flow regulator picture, so we search for this product by name.  This just executes a search on the Finder method and returns all the products which have this name.  For this product, there’s only one with this name, so we select it.  We are then returned to the edit item page and we accept the changes.  We have now mapped this one product’s picture to its matching database row.  I won’t show you, but I also mapped the other two pictures as well.  When returning to the All Pictures View, you can see that the two columns’ values were pulled from the database and stored in the list:

Figure 44: All Pictures View Now Showing External Data

Remember these values are cached in the list, so when the inventory changes, this will not be automatically reflected in the list.  Without writing code, there is no automated way to refresh this.  It’s very easy for a user to refresh the list, and you do this by clicking the refresh icon listed next to the Product Name column heading.  After clicking refresh, you are greeted with this confirmation page:

Figure 45: Updating Cached Business Data in List

In this case, updating three items is minor—if you had thousands (or more) items, it could take a while!  Security Note: If a user only has read permissions to the list, they will still see the refresh icon but will not be able to refresh its data.

Another drawback to this that you might notice is that it can be quite tedious manually mapping up each item in the list to each row in the database.  You may be able to automate some of this through code as I discuss in this blog article.

At this point, you have seen how to add Business Data columns to a SharePoint list.  Now that these are cached within the list, the data stored behaves almost like regular columns.  You can filter, sort, group, all the usual things.  Remember, though, you cannot edit through the list and have the change stored back in the database.  In other words, the synchronization is not two way.  In fact, the columns that BDC adds to the list are read only.

Wrap Up

Great job on making it all the way to the end.  You didn’t just skip down here did you?    I know this was a long article, but hopefully you were able to go through it quickly, and it gives you a strong grasp on everything you need to know with Business Data Web Parts and incorporating Business Data within your SharePoint lists.

Coming up next in Part 5, we’ll take a close look at how the search engine can crawl BDC data so that your searches encompass your business data as well.  Until then…

Why Web Services

While relational databases have proliferated greatly over the last 20 years, there are some inherent difficulties with them.  For starters, they are often proprietary.  While structured query language (SQL) is an ANSI-standard language, vendors choose to develop their own flavor of SQL such as PL-SQL with Oracle or T-SQL with Microsoft.  In order to query from a database, you need a vendor-specific driver, making integration between systems difficult.  Furthermore, the network connection is often based on remote procedure calls (or RPCs) that won’t typically work over an HTTP connection.  This makes connecting to them over the Internet or through firewalls a challenge.

Web Services have matured in the last few years as the de-facto way of integrating systems together.  Service-oriented architecture (SOA), which is based on Web Service technology, offers us a very compelling approach to building and integrating systems today.  Its loosely-coupled nature and vendor-neutral design make it far more flexible and practical for the changing needs of organizations.  Exposing line of business data over Web Services gives you more control, more universal access, HTTP compatibility, and is in line with emerging technologies such as cloud computing.

Since a primary goal of SharePoint is to be the central interface for critical data assets, it’s likely that SharePoint will need to integrate with external systems.  In article two of this series, we covered how the ADF can describe an external database.  In addition to the advantages we just covered, there may be other reasons why Web Services might be the better choice.  For example, perhaps you already have a Web Services architecture in place that you want to leverage.  Or you have data that comes from a myriad of data sources such as databases, XML files, or services in the Internet cloud.  With Web Services, your content can originate in virtually any source.  By exposing these data sources through a Web Service, you provide a single way for SharePoint to consume this data, regardless where it is located.  This is depicted in Figure 1 here:

Figure 1: Exposing Web Services to MOSS

Developing Your Web Services

Now that we have covered why Web Services might be right technology, let’s dive into a sample Web Service that we can consume from BDC.  As you can see in Figure 1 above, the data can originate from any back-end data source.  In this first example, it will come from the AdventureWorks database from a Microsoft SQL Server.  The data we want will be returned using a simple stored procedure named GetEmployees.  Here is the code for the stored procedure:

create proc [dbo].[GetEmployees]
as
select Employee.EmployeeID,
        Contact.FirstName,
        Contact.LastName,
        Employee.Title,
        Employee.HireDate
from HumanResources.Employee
join Person.Contact
on Employee.ContactID = Contact.ContactID

As you can see, we are returning four columns.  We join from the Employee to the Contact table to retrieve all the columns we need.  We then call into this stored procedure from our C#-based Web Method.  This method is coded within an ASP.NET Web Service project as shown here:

[WebMethod]
public List<Employee> GetEmployees()
{
    Database db = DatabaseFactory.CreateDatabase("Connection String");
    IDataReader employeesReader;
    List<Employee> employees = new List<Employee>();

    DbCommand cmd = db.GetStoredProcCommand("GetEmployees");

    employeesReader = db.ExecuteReader(cmd);
    while (employeesReader.Read())
    {
        employees.Add(new Employee((int)employeesReader["EmployeeID"],
                      employeesReader["FirstName"].ToString(),
                      employeesReader["LastName"].ToString(),
                      (DateTime)employeesReader["HireDate"]));
    }
    employeesReader.Close();

    return employees;
}

Version 4.1 of the data access application block from the Enterprise Library is used to return the Employee data.  This is not required as any form of ADO.NET code will work, but this simplifies our data retrieval and is a good practice.  The data returned from the Web Method is a generic list based on the Employee class.  You can consider this Employee class to be our business object representing a single employee.  Here is the part of the class with the four columns and two constructors.

public class Employee
{
    private int _id;
    private string _first;
    private string _last;
    private DateTime _hireDate;

    public int ID
    {
        get { return _id; }
        set { _id = value; }
    }
    //Additional getter/setter for remaining columns go here 

    public Employee(int id, string first, string last, DateTime hireDate)
    {
        this._id = id;
        this._first = first;
        this._last = last;
        this._hireDate = hireDate;
    }

    public Employee()
    {
        this._id = 0;
        this._first = string.Empty;
        this._last = string.Empty;
        this._hireDate = DateTime.Parse("1/1/2000");
    }
}

One key advantage of using a generic list when writing the Web Method is that it serializes nicely into XML.  It also describes the complex types in the XML Schema (XSD) in a BDC-friendly way which will make the authoring of the ADF file much easier.  When Employees are returned from the Web Method, here is how one of them will be described in XML:

Figure 2: Employee element returned from Web Service

In the Web Service, we also define two additional methods, one for an IdEnumerator and one for a Specific Finder.  For brevity sake, I only show the interface for each.  They are coded similarly, and you can download the full code at the end of this article.

public List<int> GetEmployeeIDs () //IdEnumerator 
public Employee GetEmployeeByID(int EmployeeID) //Specific Finder

GetEmployeeIDs (the IdEnumerator) just returns the ID for each Employee.  The Specific Finder returns a single Employee object based on the supplied EmployeeID.

Creating the Application Definition File

Now that we have defined a simple Web Service exposing employees, let’s look at how this is represented in our ADF.  Since the basic structure of the ADF remains the same with a Web Service as compared to a database, we’ll only focus on the areas that pertain to calling Web Services.

In Figure 3 below is our LobSystem element which is the root of the ADF.  You’ll notice that the Type attribute indicates WebService.  Inside Properties, a namespace prefix is defined for this LOB.  In our case, we call this HR (for Human Resources).  We also set the URL to the Web Service.  When this ADF is loaded into MOSS, the Web Services Description Language (WSDL) for this URL is retrieved and an internal proxy class is generated that BDC uses.  This concept is very similar when adding a Web Reference to a Visual Studio project.

Figure 3: LobSystem Root Element in ADF

Moving down in the schema we find our LobSystemInstance which is where connection details are specified.  For a Web Service, there are several options such as how to authenticate and how many simultaneous connections are allowed.  In our case, our needs are simple, so we only need to list the name of our LobSystem which is HR.  (Note: since we didn’t specify how to authenticate, this will use RevertToSelf settings which means that the application pool identity will be the credentials used if the Web Service requires authentication.)

Figure 4: LobSystemInstances in ADF

Where it gets interesting is how we start to define our methods for each entity.  The secret to writing the Web Service-based ADF is to define the TypeDescriptor elements correctly.  Each TypeDescriptor points to an element node in the Web Service XML request or response, and the basic rule is that you need to nest your TypeDescriptors to match the hierarchy.  For example, here is our Finder method that will call the GetEmployees Web Method and return a list of Employees.

Figure 5: Return Parameter for GetEmployees Web Method

Let’s go through what we’re seeing here.  First off, the Name attribute for the Method must match the name of the Web Method.  In this example, it is GetEmployees, which we introduced above.  The TypeDescriptor elements that are highlighted reflect our hierarchy in the response.  The first one defines an array of Employee elements (denoted in part by the IsCollection=”true”).  The second one defines each Employee, and the final four define each column that is returned.  In a snippet of the actual XML response shown in Figure 6 below, notice the Name attribute for the TypeDescriptor matches the Element name.  This name match-up is not required in all cases but is recommended.

Figure 6: Actual XML Response from Web Service (GetEmployees Method)

Let’s look at one more example.  Here is our IdEnumerator which returns the IDs for all Employees.  Here is the method we define in the ADF:

Figure 7: ADF Entry for IdEnumerator

In this case, we return an array of Int32 (32-bit Integers).  Below, here is the first part of the the corresponding XML that is returned from the Web Service.  Notice how the two levels in the XML, ArrayOfInt and int, align (i.e. int is a child element of ArrayOfInt).

Figure 8: Actual XML Response from Web Service (GetEmployeeIDs Method)

Consuming Employees from UI

So now that we have our Web Service ready, and our ADF loaded and secured within our SSP (as covered in the Article #2), it’s time to see the what this effort yields us in the user interface.  Let’s first take a look at the Business Data List Web Part.  We will examine this Web Part in more detail in the next article, but for now, here is a basic view of the Employee entity from our catalog:

Figure 9: Employees showing in a Business Data List Web Part

Here is a recap of how this executes:  These records are pulled in real time through a proxy class created from the ADF, through the Web Service and into the stored procedure running inside SQL Server.  Since we have defined IdEnumerator and Specific Finder methods, we can also set up a Content Source and crawl this BDC data.  This allows us to issue search queries against the external system, and this will be covered in depth in Article 5.  For now, I have set this up with the Employee entity in the ADF, and this is the result when we search for Rob (someone’s first name):

Figure 10: Search Results for Employees Based on Name

And finally, let’s look at an auto-created profile page.  This is created by the BDC that allows us to view all the details for a single record.  These types of pages will be covered in more detail in Article 8.

Figure 11: Profile Page for a Single Employee

With that, you should have a basic idea in how the ADF needs to match the response from the Web Service.  If you’ll be developing these ADFs from scratch, I advise you to look at the WSDL provided by the Web Service to understand the Complex Types in the XSD.  I also recommend looking at the actual XML data that is coming back.  From here, and with some practice, you will be able to construct your ADFs.  The process for input parameters (for example, with FilterDescriptors) is the same.

Amazon Web Service

The second example we’ll look at is a bit more complex.  Within the MOSS SDK, a sample ADF file exists to call into Amazon to do a search for a product.  This is quite powerful and great demonstration of services in the cloud.  For this section, we will look at a modified form of the sample ADF, so even if you’ve seen this, I recommend you keep reading.  Let’s first show you how this looks from the user’s perspective.  I have added a Business Data List Web Part to a web page and have pointed it to the Amazon entity in the catalog.  Here is how it shows the results when searching for SharePoint books:

Figure 12: Keyword Search Against Amazon Web Service

As you can see, it has returned the first ten titles which match these keywords.  The book’s ISBN and Title are shown, and Title links to the actual book URL on Amazon’s web site.  This was done by pulling the raw data from Amazon through the BDC and displaying it with a custom XSLT Stylesheet.

To understand how this works, let’s first understand the XML that will be delivered to BDC.  Here is what a single book (an Item) looks like in the XML Response from the Web Service (Note: I have collapsed the ItemLinks element since it’s not used in this demonstration):

Figure 13: Actual XML Response from Amazon Web Service

As you can see, we have the ISBN returned in an element called ASIN.  We also have URL stored in the DetailPageURL element.  Lastly, details about the book are stored in a child element called ItemAttributes, and this is where we need to extract the book’s title.  Taking a look at the XSD defined in the WSDL provided by Amazon, we see the definition for each Item and ItemAttributes.  (Note: I have edited the file to only focus on these two complex types).

Figure 14: A Selected Portion of Amazon’s Web Service WSDL

Now that we know what we’re dealing with in terms of schema, let’s see how the ADF comes together.  Again, I have modified the sample Amazon ADF file.  Specifically, I have removed an unused entity, and added additional logic to expose the book’s title.  Here is our single entity, called Item.

Figure 15: Item Entity Defined in ADF

Keywords is the friendly name for our filter as you can see in the screenshot in Figure 12 above.  As part of the input parameters, we provide the value of this Keyword, a subscription ID (required by Amazon to call into the Web Service), and some default values for what we’re searching (Books in this example).  These won’t be shown for brevity sake.  The concept for these input parameters is the same as the returned output in that you must define hierarchical TypeDescriptor elements to match the XML request.  With that, let’s focus on the returned response.  The hierarchy here is pretty deep, so we’ll just focus on the inner-most section that begins with an array of items.  Here is how this looks:

Figure 16: TypeDescriptor Hierarchy for Returning Items

It might look imposing at first glance, but if you follow the arrows, you can see how this maps up with the actual XML response shown in Figure 13 above.  The tricky part with this example is that we want to return Title to be at the same level as ASIN (the ISBN) and DetailPageURL.  In the XML response, Title is nested in a child node called ItemAttributes.  This becomes a problem because we need to return a flat set of columns and rows just like in a sheet in Excel.  In our case, the desired columns are ISBN, DetailPageURL and Title.  In other words, hierarchies are not allowed.  So, we somehow need to “promote” Title up one level.

We solve this problem by first setting a localized name for ItemAttributes to be Title.  This just means we rename this column to Title.  To get the actual value for Title, we use what’s called ComplexFormatting.  ComplexFormatting allows us to generate the value, and we do this using the FormatString property.  The {0} (just as with the String.Format method) will generate a new string from the value of a child element called Title.

With this described ADF, we have now effectively mapped the fairly complex result from Amazon into a pretty simple display.  If we load the ADF as is and add a Business Data List Web Part that points to this Item entity, we get the following result:

Figure 17: Raw Output of our Item Entity

Here are our three columns, ISBN, DetailPageURL and Title.  Of course, we would like the Title to just be a hyperlink to the specified URL.  We can handle this with a small change to the Web Part’s XSLT.  This is what yields the display shown in Figure 12.

Recommendations

Now that you have a pretty good idea how to structure your own, let me leave you with some recommendations that should help you construct your Web Service-based ADFs.

• The BDC only supports Simple Object Access Protocol (SOAP)-based Web Services, not REST-ful (REpresentational State Transfer) ones.
• Due to the structure of the ADF files, you will not easily be able to consume existing Web Services unless they closely match the concepts of Finder, Specific Finder, and IdEnumerator.  You might find yourself having to write specific Web Services, which is why I provided a sample on how this can work.
• If your Web Services are developed using the approach used above, you should have some success working with ADF generation tools like BDC Design Studio, BDC Meta Man, and BDC Definition Editor.  If not, you will likely need to generate the ADFs by hand.  Even when some design tools, the effort is more involved as the utility cannot infer the Identifier or FilterDescriptors for entities.  In some cases, you might get part of the ADF described using a tool, and then you’ll need to go in and make some tweaks to it.
• If you find yourself needing to write a lot of Web Service wrappers around your existing databases, you might find a utility from the same company that makes BDC Meta Man helpful.  It’s called BDC Web Man, and you can find it here.
• Keep your returned Web Service XML structures simple.  As you can see, deep hierarchies are much harder to represent in the ADF.  You’ll probably find it much easier to adjust the Web Service output than trying to craft the ADF to work with a complex structure.  For example, while you can return and consume a DataSet in the ADF, the amount of nesting makes this a very complex task, and you’ll need to code it all by hand.
• As much as possible, review the WSDL file and actual XML results when structuring the ADF file.  This will help ensure you are representing your TypeDescriptor hierarchies correctly.
• If you are developing new Web Services using .NET technologies, you can use either classic Web Services (i.e. ASMX) or the newer Windows Communications Framework (WCF) style, provided you configure it to support SOAP.
• Since your Web Service WSDL will be downloaded and parsed when you load the ADF into your SSP, make sure the Web Service is physically accessible at that time.
• Authenticating to the Web Service through HTTP or SOAP headers is supported through the BDC.  See the following article from the MOSS SDK for details on how this works.
• When troubleshooting problems with your ADF, be sure to check both the Windows Application EventLog and SharePoint’s diagnostic log files (C:\Program Files\Common Files\microsoft shared\Web Server Extensions\12\LOGS).  These will provide you with much more detail on the nature of the problem.
• As I’ve suggested, you want to review the actual XML results from your Web Services.  These can be difficult to capture at times.  To help, you might find some value in using a network sniffer such as Network Monitor.  This allows you to capture the actual network packets allowing you to study the traffic between the SharePoint server and the Web Service server.  You can download Microsoft’s Network Monitor for free from here.

Download Sample Code

In the sample code available here, you will find the following code:

• Amazon ADF (Note: you will need your own Amazon Web Service (AWS) Account)
• Modified Amazon XSLT for the Business Data List Web Part
• The complete Employee Web Service (VS.NET 2008 project)
• The Employee ADF
• Employee Stored Procedures for AdventureWorks database

ADF Overview

Since we understand the BDC to be a middle man between the user interface and some back-end system, we need some way of mapping how the back end system is structured.  This mapping can then be consumed from the various interfaces such as Web Parts and searching that we covered in the last article.  Since BDC is designed to pull data from virtually any relational database or SOAP-based Web Service, you’d correctly guess that the ADF must be flexible and generic in nature.  As mentioned already, the ADF is an XML file with a specific schema and its concept can be likened to a logical database diagram.  By logical, I mean that it does not have any vendor-specific features such as data types.  Keep in mind that it must also be flexible enough to connect to Web Services as well, so it can’t be too database-centric as well.

XML makes good sense for the definition as it is a declarative way to define the back-end in a generic way.  By being declarative, and not done programmatically through code, it helps ensure that we have a loosely-coupled design.  This is very important as front-ends and back-ends are likely to change, and we want to make sure the maintenance effort spent on this middle layer is minimized.  As we’ll see later, having it XML based also makes it easier to automatically generate this definition.

In Figure 1 below, you can see a conceptual overview of how the ADF is used.  The browser makes a regular request where BDC data is needed (e.g. a BDC Web Part, Search, etc.).  The MOSS web front end receives the request, and as part of its processing, hands the BDC request off to the Object Model. The Object Model (also known as the API or Application Programming Interface) gets the definition for this ADF from its catalog.  With the details on the ADF known, the Object Model requests the data from the database or Web Service.  The response is returned and then formatted as appropriate for delivery back to the browser.

Figure 1: Data flow overview with BDC

To best understand the ADF, it helps to have an understanding of some of the terminology used.  This will relate directly to structures found in the XML definition.  Those with a database background (and also those developers with knowledge of ADO.NET) should pick these concepts up right away.

Line of Business.  Line of business (or LOB) is just a term that refers to a back-end system.  As you’ll see shortly, a single ADF is comprised of one or more LOB instances.

Entity.  An entity works like a table.  The entity term is also used when designing a logical database diagram, and the concept is the same.

Method.  A method is how we request data (or query) from an entity.  For a database, it is usually a SQL select statement or stored procedure.  For a Web Service, it would be a Web Method.  Each entity supports multiple methods.

Parameter.  Parameters are the inputs and outputs to a method.  Each parameter has a name and a .NET data type (e.g. System.Int32 or System.String).

Filter Descriptor.  Filter descriptors provide an easy way for users to filter records (rows) found in an entity.  It is mostly commonly used with the Business Data List Web Part.  For example, you might want to allow users to quickly find a company by its name.

Association.  Associations are how relationships are represented.  For example, an Employee entity might have an association with a VacationRequest entity.  For a database, these are normally used on child tables that have a foreign key.

Action. Actions are used with entities and allow you to link to external applications or custom SharePoint pages for additional processing.  These are commonly used in write-back situations, and we will look closer at these in the final article in this series.

The ADF Schema

Now that you have a pretty good idea of what the ADF does and some of the vocabulary, let’s go ahead and examine the XML Schema for an ADF.  (Note: this is not intended to be a detailed discussion; for a deeper look, please consult the MOSS SDK (Software Development Kit) here: http://msdn.microsoft.com/en-us/library/ms563661.aspx.)  One of the screen shots from this SDK is very helpful in understanding the high-level hierarchy of the ADF.  You can see it here:

Figure 2: ADF Schema Overview

LobSystem is the root element in the ADF file.  It contains attributes to the required namespaces, specifies whether it is a Web Service or Database, and sets the Name of the ADF.

LobSystemInstance is where you define your connection in the ADF.  Even though the XML schema looks like it supports multiple connections, you can only have a single connection defined in your ADF.  Through Property child elements, you define the connection string to this data source.  Those with ADO.NET experience will quickly recognize this example below as a Windows-integrated login to a SQL Server.

Figure 3: LobSystemInstance

The details of authentication is a complex subject, and we’ll just cover the basics here.  When connecting, you have four options when it comes to authentication.

1) PassThrough.  PassThrough can be used to authenticate the user in one of two ways, using Windows credentials or database credentials. With Windows (shown in Figure 3 above), the client’s Windows credentials are forwarded to the back end system.  This can be used for both databases and Web Services.  The problem with this, however, is that unless you’re using Kerberos (or your Web Service/database server is on the same physical machine as your SharePoint web front end), this will fail.  This is due to Windows NT Lan Manager’s (NTLM) inablility to delegate.  With database credentials, you use a database-specific username and password to log in.  This can only be used against a database server, not a Web Service.

2) RevertToSelf. This means the credentials of the Application pool is used to authenticate the user.  This will solve the the delegation problem from above, but all users will effectively use the same login on the back end.

3) WindowsCredentials/RDBCredentials.  These are used in conjunction with Single Sign On (SSO).  In this model, the user’s credentials (either a Windows login or a database-specific login) for the external system are pulled from the SSO database.

4) Credentials.  This is only used if you are connecting to a Web Service and cannot use Windows authentication.

For more details on authentication, I recommend you start your reading here:  http://msdn.microsoft.com/en-us/library/ms566523.aspx.

Within LobSystemInstance, each entity for this connection is defined.  The entity itself is not very interesting as it just defines a friendly name by which users will reference it.  Within each entity, you define an identifier and methods.  An identifier is the primary key, or the column(s) from the entity that can be used to uniquely identify each row.  These are very important as they are used in a number of ways.  Often, these are ID columns, such as CustomerID as shown here.

Figure 4: Identifier

Also part of each entity are the methods.  These are really the heart of the ADF as it defines the actual data that will be pulled.  It is where you specify a select statement (e.g. select * from table), a stored procedure, or a Web Method if it’s a Web Service.  Since there may be multiple ways to query data from an entity, you can have multiple methods.  Here is a simple example with returning a few columns from a table named Customers.

Figure 5: Method

Within a method, you may want to define one or more ways in which this method can be filtered by users.  This is used within the BDC Web Parts.  For example, for the list of customers above, imagine your company has hundreds of them.  How will your users easily find the one they are looking for?  Start by asking yourself what are the common ways they might want to lookup a customer—perhaps by Name, by Region, or by a Contact Name.  If this makes sense then you understand FilterDescriptors.  Here is a simple look at two FilterDescriptor entries that have been made.  The Name attribute you see here defines the friendly name that users will work with, so make sure this name makes sense to them.

Figure 6: FilterDescriptor

Now that you understand filtering, let’s go back to methods for a bit.  In the ADF, there are different types of methods, and the three primary types we work with are Finder, SpecificFinder and IdEnumerator.  Finder methods are used when you want to return one or more rows from an entity.  For example, find all customers who are in the North America region.  SpecificFinder is used to find a single row and is used in conjunction with the Identifier mentioned above. For example, find me the one Customer that has CustomerID = 42.  IdEnumerator is used to return the identifier value for each row.  In code terms, this would look something like Select CustomerID from Customers.  The BDC uses this when crawling for Enterprise Search integration.  You will find each method type defined within a MethodInstance.

Let’s move on to Parameters.  Parameters allow you to describe the exact input and output for methods.  Input parameters (Direction = “In”) are used with Filter Descriptors.  There are different types of output parameters, but the most common are Return parameters (Direction = “Return”).  These are used to describe each column that is returned from a method.  For each parameter, you define the friendly name and its .NET-matching data type.  In Figure 7 below, we see the two parameters that are defined for our two FilterDescriptors above.  These are input parameters as shown here:

Figure 7: Parameter

Last but not least, let’s cover Associations.  We use Associations to define the actual relationships that exist in the external system.  BDC can directly work with one-to-one and one-to-many types of relationships.  When you define an Assocation, you define the SourceEntity (i.e., the parent or primary) and the DestinationEntity (i.e., the child or foreign).  You also must point to a method in the child entity that returns all the child rows based on a single parent row.  This method will usually “join” these two tables together.  In SQL terms, it would be something like

Select * from Orders
join Customers on Orders.CustomerID = Customers.CustomerID
where Orders.CustomerID = 42

Here is an example on how an Association is defined in the ADF:

Figure 8: Association

Creating ADFs

In the small snippets above, you can probably see that the XML syntax won’t likely roll off your fingertips.  It takes time and practice to get good at writing them.  These can be created in any text or XML editor, including Visual Studio.  If you will be using Visual Studio, I recommend you import the BDCMetaData.xsd schema file so at least you get IntelliSense.  You can follow the guidance on how to set this up from SharePoint MVP Andrew Connell here.

Fortunately, there are tools out there that can reverse engineer the database (or Web Service) and generate most if not all of the ADF file for you. Even if you only have to write one ADF, these tools may be worth it.  The three main ones available to you are

• BDC Definition Editor
• MOSS BDC Design Studio
• BDCMetaMan

BDC Definition Editor is free and comes from Microsoft.  You can install it after you install the MOSS SDK.  Sadly, with this tool, you get what you pay for.  It doesn’t generate Finder methods, crashes quite a lot, and requires you to install SQL Server 2005 Express (even if you have a full SQL Server instance installed).  It’s certainly better than nothing but not recommended for regular use.

MOSS BDC Design Studio is a commercial product from Simego.  The latest version 2.0 has a nice wizard interface when setting up each entity.  It can be used to generate ADFs on common databases like SQL Server and Oracle as well as Web Services. It is reasonably priced at $200 for a single license.

BDCMetaMan from Lightning Tools is the top-of-the-line and de-facto standard for generating ADFs.  It is well written and fully featured.  It is also the most expensive at $1200 per license but well worth it if you use it enough.

Loading the ADF into MOSS

Once your ADF file has been generated, it needs to be loaded into your SharePoint environment.  This is done through your Shared Services Provider (SSP).  To get there from Central Administration, click on your SSP name in the Quick Launch menu on the left, then click Import application definition in the Business Data Catalog section.  Loading it is as simple as browsing to the file and clicking the Import button.  When you upload the ADF, it will validate the XML file. It will then create entries in your SSP Database for each LobSystemInstance and Entity.

Once the ADF is loaded, you might want to take a look at its structure.  Just click on View applications from the main SSP screen, and then navigate down through your named ADF and into each entity.  With the exception of custom actions, everything else is read only, so you can’t really make any changes.

Setting Permissions

When you’re done poking around, you’ll probably want to set permissions on your new definition.  Permissions apply to these levels in the ADF hierarchy: LobSystem (the whole ADF), Entity, Method and MethodInstance.  This means you can secure at a high level such as each ADF, or very granularly such as each Method within an Entity.  Unfortunately, the administration GUI only lets you set permissions down to the Entity level, so this is the lowest level that most people work with. You can also set permissions at a higher level (even the whole BDC) and copy these down to lower levels.  When setting permissions, you have four permissions to choose from.  Each is introduced here:

• Edit.  This permission allows you to delete or add new ADFs into BDC.  You would grant this to those who would be uploading new ADFs into your SSP.
• Execute.  This allows you to execute Method Instances.  If you apply this permission at the entity level, it applies to all methods and method instances for this entity.
• Select in Clients.  This defines whether you can see the entity when configuring BDC Web Parts or adding a BDC column to a list.
• Set Permissions.  This allows you to change permissions for other users at the various levels as described in this section.

That said, the common approach is to give users both the Execute and Select in Clients permission for all entities that they will be working with.  As standard with SharePoint, you can assign permissions to Users or Groups.

With that, we bring this Part Two article to a close.  By now, you should have a solid understanding of how BDC works and how to create, load and secure your ADF entities.  In the next session, we’ll focus more on creating ADFs for Web Service systems.  From there, we’ll then move over to the UI and how to leverage the value that these ADF entries give you.

We have three goals for this series:  1) Give you a strong understanding of what BDC is and how it works; 2) Ensure you recognize the value it brings organizations; 3) Provide you with the technical know-how to set it up and use its many features.

BDC Overview

As you already know, Microsoft Office SharePoint Server (MOSS) provides tremendous value as an intranet solution. One of the ways it excels is that it provides a single access point for the data assets in your organization. In many cases, these assets are stored in a Word doc, a PDF, an Excel workbook, etc.  In other words, it’s just a file. We call these sources unstructured data due to their flexibility in storing almost endless forms of information. It’s this flexibility that has made MS Office such a popular software suite. Users can work in a rich client application and then simply save the file, presumably into a document library.

This is where the power of MOSS as a document management system comes in. When coupled with an enterprise search engine, it’s even greater. Despite the unstructured nature of these files, users can go into their portal and by either a few navigation clicks or a quick search, they can find the answers they need. Having this centralized location for answers yields not only productive employees, but happy ones. It may seem that users have the best of both worlds: the flexibility of storing data the way they want, and the ability to publish it knowing that other users can find it.

But hang on, it’s not quite this easy. Sometimes, structure is necessary. Structure imposes rules and order. Do we really want to store all our customer orders in a single Excel sheet? Structure helps ensure we have valid, manageable, and most importantly, trusted data. For these structured data needs, IT professionals have relied on the tried and true relational database, and many vendors such as Oracle, IBM and Microsoft have made a bundle selling their flavor of SQL.

With these relational database management systems, information is locked in server-side databases. You usually work with the data through a separate application, perhaps a rich client or a web-based interface. These structured databases are the lifeblood for critical systems such as accounting, CRM (Customer Relationship Management), ERP (Enterprise Resource Planning), and a myriad of other line-of-business needs.

In fact, these solutions have become so popular and easy to build, they have sprung up all over the organization into what I like to call database sprawl. Typically, each of these systems is another data island, disconnected from the others. In many cases, information is duplicated across these systems, causing chaos where order was expected. From the user’s standpoint, they can be perceived as very inflexible and frustrating: Where is the data I need?  How do I log into that system?  Once I’m logged in, how do I access it?  By the time each question is answered, precious time is lost.  Or worse, we give up and make decisions based on no data at all.

This yields an interesting dichotomy in the way we store data. On one end, we have the unstructured nature of MS Office; on the other, we have the structure provided by the relational database. Interestingly, both can result in chaos. What you’ll be pleased to know is that MOSS can also bring some order to the database sprawl as well, and it does so in a very similar way.

With Business Data Catalog, part of MOSS Enterprise, we again get directed us back to the SharePoint interface to get access to our data. With BDC, this structured data can be exposed through Web Parts; it can be indexed and searched; it can be merged with data found in lists. With BDC connections to your back-end systems, we can find and consume both structured and unstructured data assets that exist in our organization. And we do this from one location, one interface.

Architectural Overview

A holistic view of what BDC can do and how it works is best depicted in Figure 1.

Figure 1: BDC Architecture

What you see along the top are the primary ways BDC can expose data from your back-end systems. Here is an overview of each of these from left to right:

1) BDC data can be surfaced using the many out-of-the-box Web Parts. This is the most common way and very easy for the users to search, filter and sort for information. You simply add one of the BDC Web Parts to any page, point it to one of the sources in the catalog and users are ready to go. All data is pulled in real-time using this approach.

2) SharePoint Lists allows you to merge data that lives in SharePoint along with data from a back-end system. For example, perhaps you have customer names stored within a list. From your CRM system, you would also like to see contact information for each customer. By adding a BDC field to your list, you can copy the corresponding data from your back-end system and store it within the list. Typically this matchup is done one-to-one. To ensure the copied data doesn’t get stale, users can refresh it on demand.

3) Search allows you to set up a content source and point it to an application in the BDC. SharePoint then goes through through BDC to crawl your structured data source and store the keywords in its index. Your search queries can now go across both structured and unstructured sources giving you a unified set of results.

4) User Profile Importer allows you to supplement attributes found in the User Profile store from external databases. This is important for many organizations as their Active Directory or LDAP directories are rarely populated with all user details. For example, how many IT administrators will actually fill in the birthday or skills attribute? Data like this is usually found in HR systems anyway.  Using BDC, these values can be pulled and matched up with the user, providing a complete set of accurate values.

5) With custom applications, you can avoid having to write another data layer for your new application. Instead of directly connecting to the system using ADO.NET or a Web Service, you can consume the data using the BDC Object Model. This gives you a more flexible, declarative style of talking to your back end.

With each of these approaches, you may have noticed that the concept is predominantly read-only in nature. This is similar to decision support systems.  That is, we usually use BDC to consume the data from these back end systems, but rarely write back to it. In the final article in this series, you’ll see how you can write to your back end system.

A Quick Look at BDC in Action

Let’s see how this works.  In this first example, we demonstrate the Business Data List Web Part (#1 from above).  This makes the finding and displaying of rows from a table very easy.  As you see here, we are finding all rows from the Customers table (perhaps from a CRM database) that contain the word “market” in the name.

Figure 2: Business Data List Web Part

In the second example, we see how users can search for both documents and BDC sources (#3 from above).  A search for Great Lakes has been issued, and we see both document and BDC Search Results listed together.  We’ll go into far more detail on these features and others as we get into other articles.

Figure 3: BDC Search Results

How Does BDC Work

Now that you have a better idea of what it can do, you’re probably wondering how does it work? As you can see, the BDC is a middle man between your interface and the back end. To provide this universal interface, it uses a database-like XML syntax that is called an Application Definition File or ADF. This ADF defines the connection, the entities (for a database, these are usually tables, views or stored procedures), how to filter, what associations (relationships) exist and what user-friendly names should be used. These names can even be localized in different languages. This ADF is intended to be a generic interface describing the dictionary of information available. Once these ADF files are stored within the catalog, the first four options above are available without having to write any code. Of course, security is an important consideration when we talk about exposing sensitive data, so we’ll see how the BDC addresses this concern in the next article.

For the actual back end, as you can see in Figure 1 above, this can be any ADO.NET accessible database with an OLEDB or ODBC driver.  This covers most every database from MS Access and SQL Server to Oracle and IBM DB2. The back-end can also be a specially-written SOAP-based Web service.  The BDC is configured within a Shared Services Provider (SSP) which allows you to take advantage of BDC technology within any Web Applications within your SharePoint Farm.

Series Outline

With that, that nearly brings us to a close for this first article. But before we let you go, here is an outline on how we are structuring this series.  To get the full value, you should go through these in order, but we’ll try to keep them discrete so that you can jump around and focus on the ones that are most important to you. Note: these may slightly change as we go through each of these.

Part 1
In this first part, as you have now read, the focus is be on the concepts, the value it can bring organizations, the architecture, and an overview of each article in the series.

Part 2
This article will introduce the Application Definition File. By the end, you’ll understand the role it plays, how it works, and have a high-level grasp of its schema.  We’ll also introduce tools that allow you to generate these XML files.  Securing ADFs will also be covered in this article.

Part 3
This article will cover how the Web Services can be used instead of a direct connection to a database.  It will look at the kind of Web methods you’ll need to write if you want to expose your systems to BDC this way.

Part 4
This will be a two-part article.  In first half, we’ll look at the out of the box Web Parts that BDC provides.  These is the most common way that this structured data is presented to users, so we’ll give it a solid overview.  The second half will cover how you can cache BDC data into SharePoint lists.  This will allow you to marry SharePoint data and BDC together into one consolidated view.

Part 5
This article will cover everything you need to know to to configure Enterprise Search to crawl BDC data.  We’ll cover content sources, scopes, and how to configure MOSS for both full and incremental crawls of your structured data.

Part 6
As mentioned above, BDC can supplement properties contained within User Profiles. This allows you to pull critical data elements from external ERP or HR systems and map them to out-of-the-box or custom User Profile properties.  This article will cover everything you need to know to get this working for you.

Part 7
This article will apply mostly to the developers out there and will cover the BDC Object Model.  You’ll see how custom Web Parts, application pages or other custom-built SharePoint applications can consume BDC database without your having to write another data layer.

Part 8
In the final article, we will cover Custom Actions how these can be used to safely write changes to the back-end database or Web Service.  While not used often, this may prove to be a practical way for users to maintain some of the data in these external systems.

Custom Search Page Creation

The out-of-the-box Search Center actually contains both a Search Page and a Search Results page for All Sites and People. Therefore, to be consistent, we should also create a Search Page and Search Results Page for our new Clients scope.

First we will create the Search Page. Go to the Search Center site and select Create Page from the Site Actions menu:

On the Create Page screen enter Clients for the Title of the page and the URL name. Select the Search Page layout and then click on Create:

The new page will appear in Design mode. Edit the Search Box web part and enter “ClientResults.aspx” in the Target search results page URL setting (under Miscellaneous):

Click OK and Publish the page.

Creating the Search Results Page

The creation of the search results page is the same process. Go to the Search Center site and select Create Page from the Site Actions menu:

On the Create Page screen enter Client Results as the Title of the page and ClientResults for URL name. Select the Search Results Page layout and then click on Create:

The new page will appear in Design mode. Edit the Search Box web part and enter “ClientResults.aspx” in the Target search results page URL setting (under Miscellaneous):

Click OK and Publish the page.

Creating New Tabs

Go back to the main Search Center again and Edit the Page. Click on Add Tab near the top:

This will bring up a screen to add a new tab (for Search Pages). Enter Clients for the tab name and Clients.aspx as the page (remember this is for the search pages, the results are next):

Click Ok and publish the page.

Perform a search under the All Sites scope such that the results page appears. Edit this page and click on the Add New Tab towards the left of the screen:

This will bring up a screen to add a new tab again but this time it will be for the Search Results Pages. Enter Clients for the tab name and ClientResults.aspx as the page:

So now the Search Center should be loaded up with a Client Search Page, Client Search Results Page, and the appropriate tabs.

Updating the Client Scope

Now that we have the client results page created, we need to go back and update the scope such that any search within the client scope uses the proper results page. Therefore, back in Central Administration under the Search Settings, click on the View Scopes within the Scopes section:

On the next screen, select the Clients scope to bring up the scope properties and then click on the Change Scope Settings link:

This will bring up the Edit Scope screen. Simply change the Target Results Page setting to the “Specify a different page for searching this scope” and enter “ClientResults.aspx” within the Target results page box:

Click OK. The scope will need to be updated but that happens automatically.

What’s Next?

We are coming down the home-stretch. Everything is now set-up for the customization of the search results. The final article will go into details in how we can use the managed properties and the results XSL to generate customized search results within our clients scope.

Now that the BDC source has been created and crawled, managed properties must be mapped to the crawled properties such that those columns may be used within the search results (XSLT). The steps in this article need to be repeated for each data element that is to be used within the search results. For scenario purposes, a Client URL is being provided by the BDC and this is the link we want to show in the search results (versus the standard BDC properties page).

From the Configure Search Settings page and select the “Metadata property mappings” link:

On the Metadata Property Mappings page click the New Managed Property button:

On the New Managed Property page, enter a name for the property, an applicable description, and select the type of information (most values will probably be the default of Text). After this has been entered, click on the Add Mapping button:

On the Crawled Property Mapping dialog, select the Business Data category. The list of available properties should appear. Find the appropriate property and select it. (If there are too many properties, enter a part of the name and use the Find button to narrow down the list).

Click OK.  Back on the New Managed Property screen, select the “Allow this property to be used in scopes” checkbox and then click OK.

Repeat for the remaining BDC data elements.

A full crawl will be needed on the BDC content source once the managed properties have been added.

What’s Next?

This was a short and sweet discussion but very necessary to continue. The next effort is to create the custom results page for use within the client search scope and therefore the next article will show the steps to successfully accomplish this task.

The out-of-the-box search features of MOSS 2007 include two scopes, All Sites and People; but what about customers or clients? Many organizations are in business because of their customer base; so when adding new scopes and search content, the customers are the next best bet (pun definitely intended).

Therefore this article will demonstrate the process of creating a Client scope for use in search from a BDC source. It assumes that a Client BDC definition file has been created and imported into the MOSS implementation. The customer/client is simply a business scenario and the steps outlined can be used for any BDC application.

Creating a Content Source for the Client BDC

The first step is to add a content source to the search settings for the Client BDC. From Central Administration, select the SSP under the Shared Administrative Services menu. Select the Search Settings link:

On the Configure Search Settings screen, select the “Content sources and crawl schedules” link:

On the Manage Content Sources screen, select the New Content Source button:

The Add Content Source screen will appear. Enter a name for the content source and select Business Data as the type. The Applications section will refresh. Select the “Crawl selected applications” and make sure the instance checkbox is checked:

Enter the appropriate Full and Incremental Crawl schedules as applicable.

At the bottom of the screen, select the “Start full crawl…” checkbox and then click OK:

The Manage Content Sources screen should appear with the new content source entry:

Before proceeding, make sure the full crawl is completed and the status is Idle.

Adding the Client Scope

Return to the Configure Search Settings page and select the “View Scopes” link under the Scopes section:

On the View Scopes page, click on the New Scope button:

On the New Scopes page, enter a title for the scope. For now, make sure the “Use the default Search Results Page” is selected. We will generate a new search results page in future articles.

Click OK, the screen should return the View Scopes page. The new scope should be listed but with an Empty status. Click on the Add Rules link under this status:

On the Add Scope Rule page, select Content Source. This will make the Content Source section appear. Select the BDC content source created for the search previously. Leave the Behavior to the default “Include” and click OK:

Return to the Configure Search Settings screen and select the “Start update now” under the Scopes section:

The scope should begin the update process. Refreshing the screen will show the % complete on the Update status line. We are done with the SSP settings at this point.

Adding the Scope to the Site Collection

Open a new browser and navigate to the main MOSS web site. Select Site Settings from the Site Actions menus (if there is a extended menu, select “Modify All Site Settings”).

Under the Site Collection Administration section click on the Search Scopes link:

On the View Scopes page, select the Search Dropdown link:

On the Edit Scope Display Group screen, check the checkbox next to Clients scope. Click OK.

Back on the View Scopes page, select the Advanced Search link:

On the Edit Scope Display Group screen, check the checkbox next to Clients scope. Click OK.

Navigate once again to the main MOSS site. The Clients scope should now appear in the search drop-down box:

Searching within this scope should yield results from the BDC.

What’s Next?

Now that the content source and scope has been created, in order to customize the search results, managed properties need to be mapped to the crawled content. The next article will explain the steps to create the managed properties.