SQL University SSIS : Named Pipes -- Consumer

 Welcome to the third and last day of the Named Pipes seminar.  Today we will get into the consumer portion of the framework.  We will focus on getting a piece of information from the producer to the consumer and then moving some of that information into the Data Flow.

The basic layout of a consumer will be a self-contained entity.  The theory is you build it to not only interact with the producer, but to work on that single unit of work that will be passed to it.  Here is the basic layout of a consumer.  All you need to do is encapsulate all of your work in a For Loop, which controls if you have work to do.  To keep things neat, I wrap everything inside a SequenceContainer.

SQL University SSIS : Named Pipes -- Producer

Yesterday, I set up the premise for using named pipes in SSIS.  Today we are going to get into the actual coding required to get this system up and running.  In a bit I will go over how you modify your Script objects to enable you to access the libraries needed for named pipes.  Both your producer and consumer scripts need to make the changes below.   We will start with the producer Script Task in my example.

SQL University SSIS : Named Pipes -- Fundamentals

There comes a time when you look at how something is being done and say, "Does it have to be that way?".  SSIS has a few stock Data Flow Sources and they meet 90% of our needs.  There is a situation where I have a single source of data, but I would like to have parallel paths of processing.  This would be akin to the producer/consumer model.  I would like to have a single producer and multiple consumers.  The wrinkle is this data source has no way of managing itself, like a directory listing.

SQL-U Day Four Advanced SSIS : Enhanced file importing

Today we are going to do something a little different.  We are going to dig into a more efficient and powerful way to scan directories for files and process them.  The idea behind this method is to load the full path name of the files (and any ancillary data) into a SQL Server table and then build multiple Data Flow tasks to extract and process them.  We will look at a script that will do this for you as well as a sample Data Flow task that shows how to use it.  Gloves and goggles, people.  This isn't any less messy.

SQL-U Day Three Advanced SSIS : Custom destination components

 Today we are going to learn how to make a custom destination component.  Actually, how to make one work.  This isn't nearly as involved or complex as a custom source as there are fewer moving parts.  

Before we look at an example of one, here is a picture for review.  The color codes are to differentiate the two types of destination components that we are going to take apart.  And you thought you were only going to look at 1!!