sql record counts: ssis

Showing posts with label ssis. Show all posts

Thursday, March 29, 2012

difference between the fuzzy lookup and fuzzy grouping in ssis

Dear Friends,

i think fuzzy lookup

COMPARES WHAT WE ARE MAPING THE COLUMNS WITH SPELLING (IT WILL REJECT ATLEAST 1 LETTER IS DIFFRENT IN ANY RECORD MAPPED COLUMN) EX: RAVI != REVI

what is fuzzy grouping ? please explain

regards

koti

Fuzzy grouping lets you find patterns in your data that could represent duplicated data. i.e., it could match address data like "1234 Main St." to "1234 Main Street" and push these into one record. Fuzzy lookup helps you to clean dirty data by comparing your input with an outside source.

|||

Koti,

I'm just noticing your other threads here... If you have any further questions about what a component does, you should first try to find out about it through the books on-line feature of sql. Most of these components you have asked about have rather good documentation...

Please, look for the documentation BEFORE posting questions like this...

Thanks for your time.

Eric

Difference between SSIS and Biztalk

Can anybody please tell me the basic differences between Biztalk and SSIS and when to choose which technology over other?

Regards,

Sandeep Saran

SSIS is just a small piece of BizTalk. BizTalk is more Business to Business (B2B). SSIS is more simply, data integration.

I suggest you read the documentation for each on Microsoft's site to make your own judgments:
BizTalk
SSIS|||

I asked this question from a Microsoft tech evangelist and I will try and paraphrase the reply:

Biztalk is a "messaging" application basically. It is designed and optomized to move single transactions (called "messages", and consisting of relatively tiny amounts of data in each discrete "message") between different systems or processes in real time. It wraps all user data in XML. As a result, it is extremely verbose during the data movement. It is not fast and would not be very satisfactory to manipulate large data sets using batch processing. Performance would be abysmal most likely.

Whereas SSIS _is_ designed specifically to move and manipulate large datasets using extremely high performance batch processing...

|||

kenambrose wrote:
I asked this question from a Microsoft tech evangelist and I will try and paraphrase the reply:
Biztalk is a "messaging" application basically. It is designed and optomized to move single transactions (called "messages", and consisting of relatively tiny amounts of data in each discrete "message") between different systems or processes in real time. It wraps all user data in XML. As a result, it is extremely verbose during the data movement. It is not fast and would not be very satisfactory to manipulate large data sets using batch processing. Performance would be abysmal most likely.
Whereas SSIS _is_ designed specifically to move and manipulate large datasets using extremely high performance batch processing...

Think EDI - Electronic Data Interchange|||

Sandeep Saran wrote:

Hi
Can anybody please tell me the basic differences between Biztalk and SSIS and when to choose which technology over other?

Start with this whitepaper describing possible approaches:

http://www.microsoft.com/technet/prodtechnol/biztalk/2004/whitepapers/integration.mspx

Wednesday, March 7, 2012

devenv.exe hangs up with alomost 100% cpu usage

For some reason my visual studio with SSIS project hangs up with almost 100% cpu usage sometimes immediately after opening the studio.

Any resolution?

Thanks

I'm having the same problem and it is driving me nuts! I do a lot of BI work and this has disrupted my project timelines significantly.

As best as I can tell. it seems to have start after applying a patch to the .NET framework 2.0. The patch is defined in the KB922770. I'm not sure if this is the culprit by itself or if there are others factors re-enforcing the problem.

Let me know if you come up with a resolution.

Thanks, Darryl

darryl_jewett@.msn.com

|||

We were able to reduce the occurrence of hangup by 90% with two settings... particulary with the first one.

1. Project property -> Configuration Manger - > Build = UNCHECK (This gets checked mysteriously even though we Uncheck and save in TFS - so we verify for this setting frequently when we open the project). If it is checked - all packages in the project are built/compiled irrespective of package you are working on and if there are 100's of packages - compiling all of them takes hell of time.

2. Tools->Options->Environment->AutoRecovery - Save autorecovery information... = UNCHECK. (I did this bz I saw in the status bar of VS the message: Saving autorecovery details.. when it gets stuck)

Hope it helps you as well (if above setting are not important to you ).

But the problem has not gone altogether.... it still hangs up once in a while.

Thanks

Development Issues with SSIS

We have been facing some delevelopment issues with SSIS. These are as follows:

We want to execute some tasks in a particular package depending on a condition. Example: If record count is more than 0 then go to next task else stop the package.
We tried to write a script, in which we evaluate the value of the variable. This variable is populated with the record count value.
We cannot find any particular function which can exit the package successfully. We can fail the Task, but that means the package is failed. wo dont want to fail the package.
In short we want to conditionally execute some tasks in a package. Something like an "If container" similar to "For container".

We have one main package from which we execute 5 other packages. We want to assign values for connection information and variables in this main package only. The other 5 packages should pick values from this main package.

We are scheduling our package from Windows Scheduled Tasks. When the package is run in this manner, it doesnt read values of variables and connections from configuration file. It uses the values that were provided during development which are embedded in the code.

Thanks

Anup

1. You can do this in control flow by editing the precedence constraints (the arrows) and using an evaluate operation 'expression and constraint' where the expression will be based on the variable 'recordcount'

2. use package configuration for your connection manager in the children packages to 'receive' the connection string from the parent package via variable.

3. not sure about that. I recomment to use SQL Agent Job instead.

Rafael Salas

|||

Hello Anup,

To pass connection information and value of a variables to the child packages, You have to define the connection information and the value of a variable in an configuration file and later you have to create variables at the master package level to which you want the values to be passed to the child packages(note: the name of variable created in the master package should be same in all the child packages as well). And later on go to the child packages, click on package configuration and select type of "Parent Package Variable". Make sure the variable names are same as the master package variables. The value of the variables in the child packages which are hard coded will be overwritten by the values assigned from the master package. Hope this help you.

Regards,

Raju

Saturday, February 25, 2012

developing ssis in 64bit sql server 2005

I have designed a packege using my local machine ( 32bit) and am trying to deploy this package to 64bit machine and open it up.. are there any issues that I need to aware of? 32 bit ssis and 64bit ssis are comparable each other?

If you are using connections that only have 32bit drivers you will have some issues when you try to execute the package (ie: excel). Other than that you won't see any differences, besides performance gains (most likely).

|||

Hi safd*,

Apart from that,

We do have list of Oracle Connectivity issues we need to be aware of, Indeed an adventure to some, see here .

I've summarized the settings we need to be aware of in my blog .

For detailed information download Connectivity and SSIS.doc from Microsoft official site.

Thanks

Subhash Subramanyam

|||I have developed 32bit SSIS package and trying to deploy in 64bit machine ( 64bit sql server 2005). Are there any know issues that i need to aware of? are they fully comparable?|||

I'm confused, you posted the same question three days ago and you marked the reply I gave you as answered. Are you have some issues?

|||

Anthony Martin wrote:

I'm confused, you posted the same question three days ago and you marked the reply I gave you as answered. Are you have some issues?

safddddddddddddddddddddd ,

Double posting questions does not do any good to the dynamic of the forum. I am merging this thread to the original one.

Thanks Anthony, for pointing that out.
|||

All,

I am trying to execute a ssis package that has OLE DB orale connection( 32bit) in SQL server 2005 machine ( run as a package )...

I've tried

"D:\Program Files\Microsoft SQL Server (x86)\90\DTS\Binn\DTExec.exe" /FILE "C:\SSISPackage\test3.dtsx" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING E

to run the package in 32bit mode.. but still getting

Description: The connection manager "oracleconnection" will not acquire a connection because the package OfflineMode property is TRUE. When the OfflineMode is TRUE, connections cannot be acquired. End Error Error: 2007-09-20 13:47:12.61 Code: 0xC020801C Source: Data Flow Task OLE DB Source [1] Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "oracleconnection" failed with error code 0xC0014019. There may be error messages posted before this with more information on why the AcquireConnection method call failed. End Error Error: 2007-09-20 13:47:12.61 Code: 0xC004701A Source: Data Flow Task DTS.Pipeline Description... Process Exit Code 1. The step failed.

can you please help!!!

|||The last time we encountered this, reinstalling the SQL client

workstation tools fixed the issue.

|||

thanks jwelch..

Is there any way you can tell me the step by step procedure that I have to take to make OLE DB for oracle work? I have read a couple of posts and it made me more confused..

what kind of drive do i really need, what kind of patch do i have to apply .. and what configurations do i have to change?

|||Unfortunately, it's been a while, so I don't really have step by step documentation.|||

i understand i need to usine 32bit DTExec.exe to run the pakage that has oracle ole db data source..

i've tried .. but it still giving me an error... whatelse should i try ?

|||If you are still getting the "work offline" error meesage, first make sure the package isn't set to Work Offline. If it isn't, then reinstalling the client tools is the only resolution I am aware of.|||

thanks Jwelch..

where do i set the "Work offline" property in package?

|||Under the SSIS menu in BIDS, there is an option that says "Work Offline". Make sure it isn't checked.

Developing new Task in SSIS

Hi All:

For making a custom Task in SSIS, is it possible to reuse the existing code base? For e.g. If i need to append some functionalities to LookUp Transform. Can I inherit the lookUp transform class?

Thanks,

Vipul

Vipul123 wrote:

Hi All:
For making a custom Task in SSIS, is it possible to reuse the existing code base? For e.g. If i need to append some functionalities to LookUp Transform. Can I inherit the lookUp transform class?
Thanks,
Vipul

No. Unfortunately not.

-Jamie

developing custom components by extending the SSIS stock data flow components

Everything I've read says that custom data flow components are built by inheriting from the Microsoft.SqlServer.Dts.Pipeline.PipelineComponent class.

But the stock components such as the Derived Column data flow transformation must each be implemented by their own class. So how do I base my custom components on those classes? The documentation for the PipelineComponent class doesn't list any such subclasses.

Not possible unfortunately. Those classes are not overrideable.

There is functionality hopefully coming in the next version which may mean this is possible (although via a slightly different method).

-Jamie

Developer v/s Enterprise (SSIS)

assuming your asking about the differences between the 2 versions.

The following link gives an overview. The Integration services section is in the middle.

http://www.microsoft.com/sql/prodinfo/features/compare-features.mspx

hope that helps

Sunday, February 19, 2012

Developer Edition - XP Home - SSIS Import/Export?

Hi,
I have a PC with XP home installed and want to use SSIS to import text
files to the SQL 2005 at my webhost, can the developer edition of SQL
2005 install SSIS Integration services to do the import?, or will that
checkbox be greyed out when installing?Hi
Integration services is installable with developer edition, therefore it is
not the SQL Server version that is stopping this being installed. Integration
Services is not available with SQL Express. You may want to try adding it
from the command prompt see
http://msdn2.microsoft.com/en-us/library/ms144259.aspx#installationscripts
and use SQL_DTS for the component.
John
"BradMcA" wrote:
> Hi,
> I have a PC with XP home installed and want to use SSIS to import text
> files to the SQL 2005 at my webhost, can the developer edition of SQL
> 2005 install SSIS Integration services to do the import?, or will that
> checkbox be greyed out when installing?
>|||Hi,
Thanks for the reply, the problem is the OS not the version of SQL as I
see it, I know that the enterprise version wont work entirely with XP
home, I can install the client tools and get BIDS on the system, but
when running import/export wizard I get that the status of the install
isnt sufficient.. i.e. integration services wont work fully with xp
home.. im wondering if the outcome will be more successful if I install
just the developer version... will I get true full functionality.
John Bell wrote:
> Hi
> Integration services is installable with developer edition, therefore it is
> not the SQL Server version that is stopping this being installed. Integration
> Services is not available with SQL Express. You may want to try adding it
> from the command prompt see
> http://msdn2.microsoft.com/en-us/library/ms144259.aspx#installationscripts
> and use SQL_DTS for the component.
> John
>
> "BradMcA" wrote:
> > Hi,
> > I have a PC with XP home installed and want to use SSIS to import text
> > files to the SQL 2005 at my webhost, can the developer edition of SQL
> > 2005 install SSIS Integration services to do the import?, or will that
> > checkbox be greyed out when installing?
> >
> >|||Hi Brad
XP Home is not an OS I use, so I can't really say.
John
"Brad" wrote:
> Hi,
> Thanks for the reply, the problem is the OS not the version of SQL as I
> see it, I know that the enterprise version wont work entirely with XP
> home, I can install the client tools and get BIDS on the system, but
> when running import/export wizard I get that the status of the install
> isnt sufficient.. i.e. integration services wont work fully with xp
> home.. im wondering if the outcome will be more successful if I install
> just the developer version... will I get true full functionality.
>
> John Bell wrote:
> > Hi
> >
> > Integration services is installable with developer edition, therefore it is
> > not the SQL Server version that is stopping this being installed. Integration
> > Services is not available with SQL Express. You may want to try adding it
> > from the command prompt see
> > http://msdn2.microsoft.com/en-us/library/ms144259.aspx#installationscripts
> > and use SQL_DTS for the component.
> >
> > John
> >
> >
> >
> > "BradMcA" wrote:
> >
> > > Hi,
> > > I have a PC with XP home installed and want to use SSIS to import text
> > > files to the SQL 2005 at my webhost, can the developer edition of SQL
> > > 2005 install SSIS Integration services to do the import?, or will that
> > > checkbox be greyed out when installing?
> > >
> > >
>

Thursday, March 29, 2012

difference between the fuzzy lookup and fuzzy grouping in ssis

Difference between SSIS and Biztalk

Wednesday, March 7, 2012

devenv.exe hangs up with alomost 100% cpu usage

Development Issues with SSIS

Saturday, February 25, 2012

developing ssis in 64bit sql server 2005

Developing new Task in SSIS

developing custom components by extending the SSIS stock data flow components

Developer v/s Enterprise (SSIS)

Sunday, February 19, 2012

Developer Edition - XP Home - SSIS Import/Export?

Developer Edition - XP Home - SSIS Import/Export?

Developer Edition - XP Home - SSIS Import/Export?

sql record counts

Blog Archive

About Me