code.commongroove.com » MS SQL Server

SSIS: Use Two OLE DB Destination Adapters to Catch Insertion/Constraint Errors

Greg — Fri, 16 Sep 2011 15:27:27 +0000

If you are using SQL Server Integration Server (SSIS) packages to insert large volumes of data, you have undoubtedly encountered incorrectly formatted or typed data in your data source including precision errors, conversion errors, and primary key/foreign key constraint errors etc.

To avoid package failure, you can choose to redirect error output to an alternate destination rather. However, if you are using “Table or view – fast load” as your insertion method, it can be difficult to see which rows are actually the offenders in your redirected output.

One way to get around this is to use two OLE DB Destination adapters with different table access modes and then redirect the output.

On the first OLD DB Destination adapter, choose “Table or view – fast load” as the access mode, but set the maximum commit size to a specific number appropriate for your data set size. Remember the smaller the commit size the slower the package will run. In my particular case, I found a commit size of 5000 was appropriate for the data source.

Next, connect the error output from your first OLE DB Destination adapter to a second OLE DB Destination adapter rather than directly to a “bad input” file or database destination. On this adapter, choose “Table or view” as the access mode which will insert records into the database one-by-one (another reason why you will have to tune your first adapter’s commit size to get optimal balance between performance and error handling).

Finally, take the error output from this second OLE DB Destination adapter and connect it to your “bad input” destination.

When it runs, the data batch that has the bad data on the first adapter will get redirected to the second adapter rather than having the whole commit size set being dumped to your error output file. And because it is inserting one-by-one with “Table or view” access, you will only get the “real” errors dumped out to your error destination.

MS SQL Server: Quick T-SQL to Delete All User Table Data

Greg — Tue, 29 Mar 2011 15:28:20 +0000

Here’s a quick (hack) way to delete all user table data from a Microsoft SQL Server database using the undocumented procedure sp_MSforeachtable. As the name hints, this procedure repeats a SQL statement against all user tables within a database.

This example assumes all identity columns are seeded at 1. You may not be able to do that in your environment.

Also note, you could use TRUNCATE TABLE, but only if you no foreign key constraints on your tables.

EXEC sp_MSforeachtable 'ALTER TABLE ? NOCHECK CONSTRAINT ALL'
EXEC sp_MSforeachtable 'DELETE ?'
EXEC sp_MSforeachtable 'ALTER TABLE ? CHECK CONSTRAINT ALL'
EXEC sp_MSforeachtable 'IF (OBJECTPROPERTY(object_id(''?''), ''TableHasIdentity'') = 1) DBCC CHECKIDENT(''?'',RESEED,1)'

Hope this helps!

Creating a SQL CLR User Defined Function

Greg — Mon, 07 Mar 2011 20:28:07 +0000

Starting with SQL Server 2005, Microsoft added the awesome ability to reference .Net assemblies from your T-SQL procedures. Here’s a quick 5 step overview on how to get up and running with your code-based User Defined Function.

1) Create a project from the Visual C# SQL CLR Database Project template (just so you can get the correct references and get a test harness going).

2) Add a class containing your UDFs. This one does a basic Regular Expression comparison:

public class MyUdf
{
[Microsoft.SqlServer.Server.SqlFunction(IsDeterministic = true)]
public static SqlBoolean FnRegExMatch(string source, string pattern)
{
Match m = Regex.Match(source, pattern,
RegexOptions.CultureInvariant |
RegexOptions.IgnoreCase |
RegexOptions.Singleline
);
return m.Success;
}
}

3) Enable the CLR on your SQL Server instance through the configuration options:

EXEC sp_configure 'show advanced options' , '1'
GO
RECONFIGURE
GO
EXEC sp_configure 'clr enabled' , '1'
GO
RECONFIGURE
GO
EXEC sp_configure 'show advanced options' , '0';
GO

4) Add the assembly and then create a reference to your UDF:

CREATE ASSEMBLY [MyUdfs]
AUTHORIZATION [dbo]
FROM 'C:\SqlServerClr\MyUdf.dll'
WITH PERMISSION_SET = SAFE -- we only have "safe" calls in our lib
GO

CREATE FUNCTION [dbo].[FnRegExMatch](
@SOURCE nvarchar(4000),
@pattern nvarchar(4000)
)
RETURNS BIT
WITH EXECUTE AS CALLER
AS
EXTERNAL NAME [MyUdf].[FnRegExMatch]
GO

5) Now you should be able to call your function:

SELECT dbo.FnRegExMatch('my test string', '\btest\b')

Of course this is just the barest of essentials to get you going with SQL CLR UDFs, but I hope it helps!

T-SQL: Backup All User Databases

Greg — Wed, 18 Aug 2010 14:35:07 +0000

Here’s an easy, generic way to back up all user databases on a SQL Server instance:

DECLARE @name VARCHAR(50) -- database name
DECLARE @path VARCHAR(256) -- path for backup files
DECLARE @fileName VARCHAR(256) -- filename for backup
DECLARE @fileDate VARCHAR(20) -- used for file name

SET @path = 'D:\localBackups\' -- has to be a local path

SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)

DECLARE db_cursor CURSOR FOR
SELECT name
FROM master.dbo.sysdatabases
WHERE name NOT IN ('master','model','msdb','tempdb')

OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @name

WHILE @@FETCH_STATUS = 0
BEGIN
SET @fileName = @path + @name + '_' + @fileDate + '.BAK'
BACKUP DATABASE @name TO DISK = @fileName

FETCH NEXT FROM db_cursor INTO @name
END

CLOSE db_cursor
DEALLOCATE db_cursor

Hope this helps!

Shrinking your Transaction Log with SQL Server 2008 to Free Disk Space

Greg — Fri, 05 Feb 2010 00:02:53 +0000

If you have been used to clearing up disk space on your development and test SQL server instance file systems with SQL Server 2000 or SQL Server 2005 by using the famous TRUNCATEONLY option on your transaction logs, you might be disheartened that they have removed this option from SQL Server 2008 (with good reason some might say).

If you do try to use it, you will get the infamous:

‘TRUNCATEONLY’ is not a recognized BACKUP option.

To achieve the same effect with SQL Server 2008, you can toggle the recovery mode for the target database and then call your DBCC SHRINKFILE to clear up disk space hogged by your transaction log.

USE MyDatabase
GO
ALTER DATABASE MyDatabase SET RECOVERY SIMPLE
GO
DBCC SHRINKFILE('MyDatabase_log',1)
GO
ALTER DATABASE MyDatabase SET RECOVERY FULL
GO

(Of course if you are already using simple recovery, you wouldn’t need to toggle.)

Hope this helps!

Don’t get caught by ANSI PADDING, VARCHAR, and trailing whitespace!

Greg — Thu, 28 Jan 2010 01:01:12 +0000

One thing to be aware of when you have VARCHAR columns and are using ANSI PADDING: trailing whitespace is trimmed and not counted in your equality ( = / <> / LIKE) statements, so you might not get the results you are looking to be returned from your query.

In the same vein, the len() function will return the same value for two strings if one has trailing whitespace. If you need to compare those two fields including trailing whitespace values, use datalength() instead.

Here’s some example T-SQL:

DECLARE @TABLE TABLE (FIELD VARCHAR(50))

INSERT @TABLE VALUES ('some text')
INSERT @TABLE VALUES ('some text ')
INSERT @TABLE VALUES (' some text')

-- returns 2 :
-- trailing spaces trimmed,
-- leading spaces count
SELECT '"' + FIELD + '"'
FROM @TABLE
WHERE FIELD = 'some text'

-- also returns 2
SELECT '"' + FIELD + '"'
FROM @TABLE
WHERE FIELD LIKE 'some text'

-- shows the difference between len()
-- and datalength()
SELECT FIELD, LEN(FIELD) AS 'Length', DATALENGTH(FIELD) AS 'DataLength'
FROM @TABLE

Hope this helps!

December 2009 IT Toolbox for TechNet Magazine

Greg — Tue, 08 Dec 2009 20:36:19 +0000

The December 2009 TechNet Magazine is on the website. Check out my IT Toolbox column here:

December 2009 TechNet Magazine IT Toolbox

In this issue I covered:

Eraser: Eradicate Sensitive Information
SQL Server Backup Pro from Red Gate: Streamline SQL Server Backups
MindManager: Map Out Ideas, Notes and Projects

Check it out and let me know what you think!

And if you have a tool you want to see me review, please suggest it to me here: tntools@microsoft.com.

T-SQL: Change Collation on your LIKE Clause Using COLLATE

Greg — Fri, 24 Jul 2009 22:17:25 +0000

Ever have trouble matching unicode letter equivalents in your T-SQL queries? For example, when you want “Québec” and “Quebec” to be equivalent in your text search query?

This is where specifying Accent Insensative Collation (The “AI” in all those collation codes) comes to the rescue. If your database is already set to an AI collation, then you are all set already, but if not, change your LIKE statement to use something like the following:

SELECT *
FROM MyTable (NOLOCK)
WHERE MyColumn COLLATE SQL_Latin1_General_CP1_CI_AI LIKE '%quebec%'

Hope this helps!

T-SQL Hack to Turn a Set of Row Values into a Delimited String

Greg — Thu, 26 Feb 2009 04:34:38 +0000

Ever want to turn multiple rows into a comma separated list when you are querying your database?

Here’s a quick way to get it done using “FOR XML PATH(”)”:

DECLARE @comma_delimited_list VARCHAR(6000)
SELECT @comma_delimited_list =
(
SELECT productName + ','
FROM Products
WHERE productName LIKE 'a%'
FOR XML PATH('')
)

IF LEN(@comma_delimited_list) > 0
SELECT @comma_delimited_list = STUFF(@comma_delimited_list ,LEN(@comma_delimited_list), 1, '')

A couple things to remember: 1) you have set your path to ” and 2) you can’t name your selection or it will put the name into your xml adding more information than you want.

Quick Method of Finding Out What Version and Service Pack your SQL Server Instance is Running

Greg — Sat, 24 Jan 2009 00:36:43 +0000

If you are looking for a quick way to verify the service pack and version applied to your SQL server instance, try the following T-SQL:

SELECT
SERVERPROPERTY('productversion') AS productversion,
SERVERPROPERTY('productlevel') AS productlevel,
SERVERPROPERTY('edition') AS edition

Hope this helps!