Friday, August 15, 2008

Microsoft Visual Studio Team System 2008 Database Edition (aka “Data Dude”) provides tools for managing and deploying SQL Server databases. 

In this article, we will discuss how to migrate data from one database to another.   Data Dude provides the Data Compare tool for this purpose.

In order to use the Data Compare tool, the following conditions must be true

1.       Data exists in a source table.  You want to migrate that data to a table of the same name in a different database.

2.       Both tables must have the same structure.

3.       Both tables must have a primary key to uniquely identify rows.

Follow the steps below to migrate data with the Data Compare tool.

1.       Launch Visual Studio 2008

2.       Select Data | Data Compare | New Data Compare.  The New Data Comparison dialog displays

  Figure 1

a.       When migrating data from a table in one database to another, the database you intend to update is known as the “Target Database”.  The other database is known as the “Source Database”.  In the New Data Comparison dialog, select the Source Database and Target Database connections.  If you have not created a Visual Studio connection to these databases in Visual Studio, you can click the New Connection button to create them.

b.      The New Data Comparison dialog contains checkboxes that allow you to specify which rows you want to see and compare.  I don’t usually change these (they are all checked by default) but it may speed up the process to clear the Identical Records checkbox.

c.       Click the Next button to advance to the second screen of the wizard.

    Figure 2
On this screen, you can choose which tables to compare.  Usually I am only interested in one or two tables, so I clear the rest of the checkboxes.  If I have a million rows in my customer table and I’m not interested in migrating any of those rows, I can save a lot of processing time by un-checking the customer table.

d.      Click the Finish button to display the Data Compare window.

3.       The Data Compare window consists of two panes: the Object List on top and the Record Details at the bottom.

    Figure 3

a.       The object list displays each table or view as a single row with columns summarizing the number of new, removed, changed and identical rows.  Rows are matched on their primary key.

b.      Click a row in the object list to display details in the record details pane.  Here you can click a tab to view the rows that are new, missing or changed.  Checking the checkbox next to a record flags it to the Data Compare tool, meaning you want to update the target database to match the same row in the source database.  This may result in an INSERT, UPDATE, or DELETE statement depending on the tab on which the record is listed. 

c.       For a record to be flagged for update, both the table and the record must be checked.

4.       After checking all the rows you wish to update, click the Write Updates toolbar button to commit your changes to the target database. 

5.       Alternatively, you can click the Export To Editor toolbar button to generate a SQL script that you can run in the SQL Server query editor.  This method requires an extra step but has the following advantages

a.       You can modify the script before running it.

b.      You can send the script to someone else to run.

c.       You can view the script to learn what Data Dude is doing.  It’s interesting to note that constraints on each table are dropped before copying data, then created after the data is copied.  This speeds up the process.  Also, note the use of transactions to prevent incomplete data copies.  Below is a sample script updating data in one table.
/*
This script was created by Visual Studio on 8/15/2008 at 8:57 AM.
Run this script on dgiard.Test_QA.dbo to make it the same as dgiard.Test_Dev.dbo.
This script performs its actions in the following order:
1. Disable foreign-key constraints.
2. Perform DELETE commands.
3. Perform UPDATE commands.
4. Perform INSERT commands.
5. Re-enable foreign-key constraints.
Please back up your target database before running this script.
*/

SET NUMERIC_ROUNDABORT OFF
GO
SET XACT_ABORT, ANSI_PADDING, ANSI_WARNINGS, CONCAT_NULL_YIELDS_NULL, ARITHABORT, QUOTED_IDENTIFIER, ANSI_NULLS ON
GO
/*Pointer used for text / image updates. This might not be needed, but is declared here just in case*/
DECLARE @pv binary(16)
BEGIN TRANSACTION
ALTER TABLE [dbo].[OrderDetails] DROP CONSTRAINT [FK_OrderDetails_Orders]
ALTER TABLE [dbo].[OrderDetails] DROP CONSTRAINT [FK_OrderDetails_Products]
ALTER TABLE [dbo].[Orders] DROP CONSTRAINT [FK_Orders_Customers]
DELETE FROM [dbo].[ProductTypes] WHERE [ProductTypeID]=N'5646953f-7b89-4862-bcf3-bf53450d28bb'
INSERT INTO [dbo].[ProductTypes] ([ProductTypeID], [ProductTypeName]) VALUES (N'7beb0d99-d034-41b9-bbf7-f9cdcdbedc30', N'Furniture')
INSERT INTO [dbo].[ProductTypes] ([ProductTypeID], [ProductTypeName]) VALUES (N'abc19a14-5968-4c5f-9f0f-4debc034cb90', N'Hardware')
INSERT INTO [dbo].[ProductTypes] ([ProductTypeID], [ProductTypeName]) VALUES (N'b9e446ed-eeb1-4334-b191-c70a55ef1a05', N'Books')
ALTER TABLE [dbo].[OrderDetails] ADD CONSTRAINT [FK_OrderDetails_Orders] FOREIGN KEY ([OrderID]) REFERENCES [dbo].[Orders] ([OrderID])
ALTER TABLE [dbo].[OrderDetails] ADD CONSTRAINT [FK_OrderDetails_Products] FOREIGN KEY ([ProductID]) REFERENCES [dbo].[Products] ([ProductID])
ALTER TABLE [dbo].[Orders] ADD CONSTRAINT [FK_Orders_Customers] FOREIGN KEY ([CustomerID]) REFERENCES [dbo].[Customers] ([CustID])
COMMIT TRANSACTION

The Data Compare tool is a simple tool for accomplishing a useful task.  Since I discovered it, it has saved me a lot of time setting up new data environments.

 

.Net | SQL Server | Tech | VSTS
Friday, August 15, 2008 8:02:53 AM (Eastern Standard Time, UTC-05:00)
 Thursday, August 14, 2008

Writing Unit Tests is an essential step in developing robust, maintainable code.  Unit Tests increase quality and mitigate the risk of future code changes.  However, relatively few developers take the time to write unit tests for their stored procedures.  The primary reason for this is that few tools exist to test stored procedures.

Microsoft Visual Studio Team System 2008 Database Edition (aka “Data Dude”) provides tools to help developers write unit tests against SQL Server stored procedures.  The tool integrates with MSTest, wich is a testing framework many developers are already using for their other unit tests.

In order to write unit tests for your stored procedures, those stored procedures must be in a database project.  For information on how to create a database project from a SQL Server database see: http://www.davidgiard.com/2008/08/11/DataDudeTutorial1CreatingADatabaseProject.aspx

This document describes how to create a database unit test.

1.       Launch Visual Studio and open your Database Project.

2.       Open the Schema View.  Select View | Schema View.

3.       Right-click a stored procedure and select Create Unit Test from the context menu.  The Create Unit Tests dialog displays.

    Figure 1

4.       Check the checkboxes next to all the stored procedures for which you wish to create unit tests.  Select the .Net language (Visual Basic .Net or C#) in which you want the automatic code to be generated.  You won’t be modifying this code so it isn’t that important, but I tend to keep all my code in the same language, so you may as well choose your favorite language here.  Enter a meaningful name for the Unit Test Project and class.  I like to name my Unit Test projects the same as my database project, followed by “Tests” or “UnitTests”.  If this is a new Database Unit Test Project, the Database Unit Test Configuration dialog displays.

    Figure 2

5.       The Database Unit Test Configuration dialog allows you to specify what you want to occur when you run these unit tests.  The dialog is organized into the following sections.

a.       Database connections

                                                               i.      Execute unit tests using the following data connection
This is the database against which tests will run.  Typically I set this to my Development or QA database.

                                                             ii.      Use a secondary data connection to validate unit tests
You may specify a different database to validate the syntax of your unit tests and test that all the objects you refer to exist.  I can only think this might be good if you are writing tests while disconnected from your testing database, but I never set this option.

b.      Deployment

                                                               i.      Automatically deploy the database project before unit tests are run
To save manual steps, you may wish to check this box and deploy the database project to the database each time you run your unit tests.  This slows down the testing step so I do not select this option.  I prefer to deploy my changes once; then run my unit tests – sometimes several times.

c.       Database state

                                                               i.      Generate Test data before Unit tests are run
It is often useful to populate your database with some test data prior to your test run.  Use this button to do this.

6.       After creating your unit tests, you need to modify each one and specify what you are testing.  Open the Solution Explorer (View | Solution Explorer).

7.       Double-click the unit test class to open it in the unit test designer. 

    Figure 2

8.       The Unit Test Designer contains some controls and two panes as described below. 

a.       A class can contain multiple tests.  The first control is a dropdown that allows you to select which test you are designing.

b.      To the right of the Test Name dropdown is another dropdown that allows you to specify what part of the test you are writing.  You can choose between the test itself, the “Pre-test” (which runs before the test is executed) and the “Post-test” (which runs after the test has completed – successfully or unsuccessfully).

   Figure 3

c.       Further to the right are three buttons that allow you to add a new test or to delete or rename the currently active test.

d.      Below the controls is the test editor.  This is where you will write your test.  You will write your test in T-SQL and Data Dude provides some stub code to get you started.  Write SQL statements that call your stored procedure and return one or more results.

e.      Below the test editor is the Test Conditions pane.  It is here that you enter your assertions. 

    Figure 4

                                                               i.      You can test for a given result set having 0 rows, 1 or more rows, or an exact number of rows. 

                                                             ii.      You can also test if a specific column and row in a given result set evaluates to a given value. 

                                                            iii.      Click the “+” button to add new assertions.  Highlight an existing assertion row and edit the row or click the “x” button to remove the assertion. 

                                                           iv.      Use the properties window to modify properties of the assertion.  Many assertions are based on a given resultset.  When I first started writing unit tests, I found it difficult to determine which resultset was which.  Basically, any line in your SQL script that begins with the word “SELECT” creates a resultset.  Each resultset is numbered, beginning with 1, in the order it is created in your script.  I sometimes find it useful to copy the SQL code and paste it into SQL Management Studio query window and run it.  Each resultset then appears in a separate grid in the Results pane.  Looking at these grids allows me to more easily see a sample result set and in what order they are created.

f.        You run your database unit tests the same ways you run any MS Test unit test.  One way to run the tests is to open the Test List Editor (Test | Windows | Test List Window), check the tests you want to run, and click the Run Checked Test toolbar button.  Tests in which all assertions prove true are passed; all others are failed.

Using Data Dude, you can extend your unit tests to cover your database objects and, therefore, improve the overall quality and maintainability of your code.

.Net | SQL Server | Tech | VSTS
Thursday, August 14, 2008 8:56:47 AM (Eastern Standard Time, UTC-05:00)
 Wednesday, August 13, 2008

Microsoft Visual Studio Team System 2008 Database Edition (aka “Data Dude”) provides tools for managing and deploying SQL Server databases. 

In our last tutorial, we described how a database developer would use the Schema Compare tool to update a database project with changes to a SQL Server database.

This article describes how to use the Schema Compare tool to push those changes out to a different SQL Server database.  There are two scenarios where you would do this. 

In Scenario 1, a developer has a local copy of the development database and wishes to get the latest updates to the database. 

In Scenario 2, a database administrator (DBA) or build master who is charged with migrating database changes from one environment to the next.  Just as .Net and web code gets regularly migrated from a development environment to a QA or Production environment, database object code must also be migrated, and that migration generally must be kept in sync with all code that depends on those database objects.

We start this process by launching Visual Studio and opening the database project.  If a source code repository such as TFS is used, we need to get the latest code from the repository.

The database to which we wish to write the changes is known to Data Dude as the “target database”.  We need to make sure that a connection exists in Visual Studio to the target database.  This is a one-time step and you can use the Server Explorer (View | Server Explorer) to create a connection.

The following steps describe how to propagate changes to the database.

1.       Launch Visual Studio and open the database project.  Get the latest source code from your source code repository.

2.       From the Visual Studio menu, select Data | Schema Compare | New Schema Compare.  The New Schema Compare dialog displays.

    Figure 1

3.       Under Source Schema, select the Project radio button and select your database project from the dropdown list.

4.       Under Target Schema, select the Database radio button and select the connection to your database from dropdown list.

5.       Click the OK button to display the Schema Compare window.

    Figure 2

6.       The Schema Compare window lists every object that exists in either the database or the database project.  The objects are grouped in folders by object type (Tables, views, stored procedures, etc.)  You can expand or collapse a folder to view or hide objects of that type.  The important column is “Update Action” which describes what will happen if you write the updates to the target.

a.       Objects that exist in the source (the project) but not in the target (the database) were likely recently added after the last synchronization.  By default, the Update Action will be “Create” meaning the object will be created in the target database.

b.      Objects that exist in both the source and the target will have an Update Action of “Update” if they have been modified in the database since the last synchronization or “Skip” if they have not.

c.       Objects that exist in the destination (the database) but not in the source (the project) were likely dropped after the last synchronization. By default, the Update Action will be “Drop” meaning the object will be removed from the database.

7.       On a database with many objects, it is useful to view only the objects that have changed since the last synchronization.  To do this, click the Filter toolbar button and select Non Skip Objects. 

    Figure 3

8.       If you wish, you can modify the Update Action on objects by selecting the dropdown in the “Update Action” column.  Some actions are grayed out because Data Dude will not allow you to perform any action that would violate referential integrity rules. 

9.       After you have set the “Update Action” of every object appropriately, you have a couple options.

a.       You can migrate your changes immediately to the target database by clicking the “Write Updates” toolbar button.  Click Yes at the confirmation to write the updates to the database project.

b.      Alternatively, you can export your changes to a SQL script by clicking the Export To Editor toolbar button.  This will create a single text file containing SQL script that you can run from a query window of SQL Server Management Studio.  This is useful if you need to make changes to the script prior to executing.  I have used this technique when my database contains views or stored procedures that refer to remote servers and I want to modify the name of the server before migrating the object.

Alternatively, you can deploy changes from a database project to a database by “Deploying” the project (select Build | Deploy Solution).  This deploys your changes using the settings found on the Build tab of the project properties page.  This method requires fewer steps, but it is less flexible than the method described above.  In particular, it does not allow you to select which objects are deployed or export and modify the script of database changes.

In the next article, we will discuss how to use Data Dude to write Unit Tests against SQL Server stored procedures.

.Net | SQL Server | Tech | VSTS
Wednesday, August 13, 2008 6:46:54 AM (Eastern Standard Time, UTC-05:00)
 Tuesday, August 12, 2008

Microsoft Visual Studio Team System 2008 Database Edition (aka “Data Dude”) provides tools for managing and deploying SQL Server databases. 

In our last tutorial, we described how to create a new database project based on an existing SQL Server database.  As the source database changes, you will want to update the database project to reflect those changes.  This article describes how to use the Schema Compare tool to import database schema changes into your database project.  The steps in this article are typically performed by a database administrator (DBA) or database developer who is charged with creating tables, views, functions and stored procedures.

The Schema Compare tool can be used to display and manage differences between two databases, between two database projects, or between a database and a database project.  Most of the time, I use it to compare a database with a database project. 

After making a change to a database schema (for example, adding a new table or adding a new column to a table), use the Schema Compare tool as described below to update an existing database project with these changes.

1.       Launch Visual Studio and open your Database Project. (For info on how to create a database project from a SQL Server database see: http://www.davidgiard.com/2008/08/11/DataDudeTutorial1CreatingADatabaseProject.aspx )

2.       From the Visual Studio menu, select Data | Schema Compare | New Schema Compare.  The New Schema Compare dialog displays.

    Figure 1

3.       Under Source Schema, select the Database radio button and select the connection to your database from dropdown list.

4.       Under Target Schema, select the Project radio button and select your database project from the dropdown list.

5.       Click the OK button to display the Schema Compare window.

    Figure 2

6.       The Schema Compare window lists every object that exists in either the database or the database project.  The objects are grouped in folders by object type (Tables, views, stored procedures, etc.)  You can expand or collapse a folder to view or hide objects of that type.  The important column is “Update Action” which describes what will happen if you write the updates to the target.

a.       Objects that exist in the source (the database) but not in the target (the project) were likely added to the database after the last synchronization.  By default, the Update Action will be “Create” meaning the object will be created in the database project.

b.      Objects that exist in both the source and the target will have an Update Action of “Update” if they have been modified in the database since the last synchronization or “Skip” if they have not.

c.       Objects that exist in the destination (the project) but not in the source (the database) were likely dropped from the database after the last synchronization. By default, the Update Action will be “Drop” meaning the object will be removed from the database project.

7.       If you are updating your database project frequently, most of the objects will be unchanged and marked “Skip”.  On a database with many objects, it is useful to view only the objects that have changed since the last synchronization.  To do this, click the Filter toolbar button and select Non Skip Objects 

    Figure 3

8.       At this point, you can view differences and you may wish to modify the Update Action of some objects.

a.       If you click on an object row in the Schema Compare window, the SQL definition code of both the source and destination version appears in the Object Definition window.  Any differences between the two versions will be highlighted (changed lines in darker blue; new lines in darker green).

b.      If you like, you can modify the Update Action any object by selecting the dropdown in the “Update Action” column.  Some actions are grayed out because Data Dude will not allow you to perform any action that would violate referential integrity rules.  If several developers are sharing the same development database, you may wish to skip those objects on which you are not working.  You may also decide that some objects are ready to share with the rest of the team and others are not fully tested and should be skipped.  It is possible to change the Update Action of every object of a given type by right-clicking the type folder and selecting the desired action to apply to all objects of that type.

9.       After you have set the “Update Action” of every object appropriately, you can migrate your changes to the database project by clicking the Write Updates toolbar button.  Click Yes at the confirmation to write the updates to the database project.

    Figure 4

10.   If you are using a source control repository, such as TFS, you will want to check in your changes.

In the next article, we will discuss how to use the Schema Compare tool to write changes to a new database environment.

.Net | SQL Server | Tech | VSTS
Tuesday, August 12, 2008 7:40:04 AM (Eastern Standard Time, UTC-05:00)
 Monday, August 11, 2008

Microsoft Visual Studio Team System 2008 Database Edition (aka “Data Dude”) provides tools for managing and deploying SQL Server databases.  In order to use Data Dude to manage an existing SQL Server database, the first step is to create a database project. 

There are a couple key points you will need to know before using Data  Dude.

1.&n