Showing posts with label columns. Show all posts
Showing posts with label columns. Show all posts

Thursday, March 29, 2012

Deserialization Failed: Parent Columns and Child Columns don't have type-matching columns.

After about an hour and a half of validation, my Analysis Services database has been migrated from 2000 to Analysis Services 2005.

I am now receiving the error in the subject any time I try and access the Data Source View that was created. All dimensions and cubes cannot be processed.

I have tried editing the xml manually to no avail.

Any ideas would be much appreciated!

Would still like to know what the issue is, but I found a workaround in the meantime.

I manually edited the .dsv file, renamed the ids, and restarted OLAP. I then opened the project in Business Intelligence Studio, and manually created the DSV. After a couple of renames of calculated columns the project seems to build properly.

|||I have exactly the same problem (Migration : all OK but I can not open the dsv properties )
I don't understand your resolution.
The migration generated a file .dsv.xml
What "Id" did you rename ?
I tried to rename the data source view ID but still cannot open
I cannot open the databases through Visual Studio. Same error message.

Deserialization Failed: Parent Columns and Child Columns don't have type-matching columns.

After about an hour and a half of validation, my Analysis Services database has been migrated from 2000 to Analysis Services 2005.

I am now receiving the error in the subject any time I try and access the Data Source View that was created. All dimensions and cubes cannot be processed.

I have tried editing the xml manually to no avail.

Any ideas would be much appreciated!

Would still like to know what the issue is, but I found a workaround in the meantime.

I manually edited the .dsv file, renamed the ids, and restarted OLAP. I then opened the project in Business Intelligence Studio, and manually created the DSV. After a couple of renames of calculated columns the project seems to build properly.

|||I have exactly the same problem (Migration : all OK but I can not open the dsv properties )
I don't understand your resolution.
The migration generated a file .dsv.xml
What "Id" did you rename ?
I tried to rename the data source view ID but still cannot open
I cannot open the databases through Visual Studio. Same error message.

Description Metadata

When creating columns in MS-SQL there is a description field. How can I quer
y
this description field? I know how to return the Table Name, Column Name,
Data Type, Is_Nullable, etc. but I don’t know how to return the Descriptio
n.> When creating columns in MS-SQL there is a description field.
This is only ifyou create the columns in Enterprise Manager *and* choose to
use that property. Personally, I don't think you should do either, mostly
because your CREATE TABLE statements should be something that you can
script, modify, store in source control, etc.
In any case:
http://www.aspfaq.com/2244|||Aaron, good point! So where would you store the description of individual
columns? Would you create separate tables to store the descriptions of the
tables and the columns?|||> Aaron, good point! So where would you store the description of individual
> columns? Would you create separate tables to store the descriptions of the
> tables and the columns?
Why do you feel the description of your database schema needs to be stored
in the database?
We create separate human readable documentation. This allows us much more
freedom in how we are describing the schema, e.g. definitions > 64
characters, diagrams, tables, etc.
A|||I was thinking of creating a web app so that the descriptions, rules, etc.
can be documented, searched, and studied by several disparate groups. As of
now the database is completely undocumented. Then to complicate matters even
more, the knowledge of this database is scattered amongst many different
groups and individuals who won’t always work together. I keep running into
columns named TX5_RRT_SDescr and no one knows what the hell that column is
used for. The DBA is too busy to answer most questions and his answers are
too flipped to be of much use.
What form in your documentation in?|||Most of ours are in Word format. But you can make your job easier using an
extraction tool, for example Enterprise Architect.
Hey, if the DBA is too busy to answer questions that need to be answered,
why do you think he'd be willing to populate some table with the answers?
"Tome73" <Tome73@.discussions.microsoft.com> wrote in message
news:200AB1D7-AD01-426A-B580-7CE42716BBDE@.microsoft.com...
>I was thinking of creating a web app so that the descriptions, rules, etc.
> can be documented, searched, and studied by several disparate groups. As
> of
> now the database is completely undocumented. Then to complicate matters
> even
> more, the knowledge of this database is scattered amongst many different
> groups and individuals who won’t always work together. I keep running
> into
> columns named TX5_RRT_SDescr and no one knows what the hell that column is
> used for. The DBA is too busy to answer most questions and his answers are
> too flipped to be of much use.
> What form in your documentation in?|||Lol, he won’t! The rest of us peons will have to collaboratively piece thi
ngs
together. There are about 5 of us who have volunteered for this project and
I
just need to figure out how to best do it. I have a zero budget to work with
.
Ok then your suggestion is not to store the description in database
properties. Thanks.
The answer to my original post is:
Where Table name is ‘Admin_Users’ and the column name is ‘User_ID’
SELECT name, CONVERT(varchar(2000), [value]) AS Description
FROM ::fn_listextendedproperty(NULL, 'user', 'dbo', 'table',
'admin_Users', 'Column', 'User_ID')|||> The answer to my original post is:
> Where Table name is ‘Admin_Users’ and the column name is ‘User_ID’
> SELECT name, CONVERT(varchar(2000), [value]) AS Description
> FROM ::fn_listextendedproperty(NULL, 'user', 'dbo', 'table',
> 'admin_Users', 'Column', 'User_ID')
Yep, I posted a link to code samples earlier:
http://www.aspfaq.com/2244|||You could store your meta data in SQL Server tables and then develop your
own custom GUI and reports around it, but there are many 3rd party
applications would do a better job.
Tools like ERWin, Viseo (or even Enterprise Managers Diagramer) can be used
to create annotated diagrams of the database model
http://www.databaseanswers.com/modelling_tools.htm
Reporting Services 2005 has a Model Designer that provides end users with a
high level view of the database model for use with Report Builder.
http://www.devx.com/dbzone/Article/28047/1954?pf=true.
"Tome73" <Tome73@.discussions.microsoft.com> wrote in message
news:200AB1D7-AD01-426A-B580-7CE42716BBDE@.microsoft.com...
>I was thinking of creating a web app so that the descriptions, rules, etc.
> can be documented, searched, and studied by several disparate groups. As
> of
> now the database is completely undocumented. Then to complicate matters
> even
> more, the knowledge of this database is scattered amongst many different
> groups and individuals who won't always work together. I keep running into
> columns named TX5_RRT_SDescr and no one knows what the hell that column is
> used for. The DBA is too busy to answer most questions and his answers are
> too flipped to be of much use.
> What form in your documentation in?|||Aaron, yes you did and I missed it at first. I found
http://developer.com/db/article.php/3361751 which is basically the same.
Hey thanks Aaron for all your help and insight.sql

Tuesday, March 27, 2012

Derived Columns in one to many relationships

I'm trying to write a query that concatenates multiple records into one
derived column. Let's say I have an author (Joe Writer) who has written
three books (Book 1, Book2 and Book 3). The author is in tblAuthors, his
books are in the tblBooks and they are joined by the AuthorID field
(number). If I use a simple select query to give me the author name and the
title, I will get three records, one for each book written.

What I want is to have all three books combined into one derived column. So
if I do the select statement, I will get one column with the author name,
and the second column will put together all three names of the book
separated by a column. So it will look like:

Author Title

Joe Writer Book 1, Book 2, Book 3,

Rather than having it appear as 3 records:

Joe Writer Book 1
Joe Writer Book 2
Joe Writer Book 3

Could someone help me with the SQL involved in this?

Thanks for the help.

Cheers,

MikeOne approach is shown in http://www.mvps.org/access/modules/mdl0008.htm at
"The Access Web"

--
Doug Steele, Microsoft Access MVP
http://I.Am/DougSteele
(no e-mails, please!)

"Big Time" <big-time-grizz@.remove-for-spam-hotmail.com> wrote in message
news:cfu2e7$18gm$1@.lettuce.bcit.ca...
> I'm trying to write a query that concatenates multiple records into one
> derived column. Let's say I have an author (Joe Writer) who has written
> three books (Book 1, Book2 and Book 3). The author is in tblAuthors, his
> books are in the tblBooks and they are joined by the AuthorID field
> (number). If I use a simple select query to give me the author name and
the
> title, I will get three records, one for each book written.
> What I want is to have all three books combined into one derived column.
So
> if I do the select statement, I will get one column with the author name,
> and the second column will put together all three names of the book
> separated by a column. So it will look like:
> Author Title
> Joe Writer Book 1, Book 2, Book 3,
> Rather than having it appear as 3 records:
> Joe Writer Book 1
> Joe Writer Book 2
> Joe Writer Book 3
> Could someone help me with the SQL involved in this?
> Thanks for the help.
> Cheers,
> Mike|||Mike,

read this article...

http://www.mvps.org/access/modules/mdl0004.htm

It has code that does this.|||Try this out
DECLARE @.BookNames varchar(1000)
SET @.BookNames = ''
SELECT @.BookNames = @.BookNames +Book + ', ' FROM Books
where author = 'Joe Writer'
SELECT 'Joe Writer',@.BookNames|||JK (jaikrishnan_nair@.hotmail.com) writes:
> Try this out
> DECLARE @.BookNames varchar(1000)
> SET @.BookNames = ''
> SELECT @.BookNames = @.BookNames +Book + ', ' FROM Books
> where author = 'Joe Writer'
> SELECT 'Joe Writer',@.BookNames

This may work. Or not work. The result of this sort of operation is
undefined in SQL Server. This is one of the few situations where
iterating over the data is a better option.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp

Derived Column Task failing with error 0xC0049067

I have a package that fails as soon as it hits the first Data Flow that contains a Derived Column task. The task takes three date columns and looks for a date of 6/6/2079. If it is there, it is replaced with a NULL. This task worked fine until I installed the Non-CTP version of SQL 2005 SP1, earlier today. (I went from RTM 9.0.1399 to SP1 9.0.2047)Does anyone have any ideas?

Here is the error I am trapping:

An error occurred while evaluating the function.

The "component "Update Max Date Value to NULL" (346)" failed because error code 0xC0049067 occurred, and the error row disposition on "input column "WeekEndingDate" (455)" specifies failure on error. An error occurred on the specified object of the specified component.

The ProcessInput method on component "Update Max Date Value to NULL" (346) failed with error code 0xC0209029. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.

Thread "WorkThread0" has exited with error code 0xC0209029.

I tried to configure the error output to ignore the failure. The task succeeded. It did not insert anything, however, for the three columns in question. It then failed on the next Data Flow that contained a Derived Column task. I tried to pre-populate the columns with a default date, but no go. Any help would be appreciated.

Nothing wrong with the concept and I can do a similar thing in SP1. What is your expression and column types?|||

The date columns come out of an Access database. I convert them to a string in a Data Conversion task and then validate that they are valid dates in a Script Component task. If they are invalid, I assign them a date of 6/6/2079. When they exit the Script task they are converted back to a date. The Derived Column task is next and it looks for the year 2079 to convert it to a NULL. Here is an example of one of the expressions:

YEAR([Check for Valid Dates].LastWorkedDate) == 2079 ? NULL(DT_DBTIMESTAMP) : [Check for Valid Dates].LastWorkedDate

This is where it fails. One thing I failed to mention before is that if I execute the task by itself it works properly. However, if I run the entire package (100+ tasks) it fails as soon as it hits the first task that contains a Derived Column task.

|||Turns out it is not the first Data Flow with a Derived Column task that is failing. I have one about 8 steps prior that runs just fine. I am wondering if it has something to do with the expression language. I saw a post where someone got the same error if they tried to use an expression language date function with a date value outside of SQL range (1/1/1753 to 12/31/9999). This post however was pre-SP1. I wonder whether that issue was addressed and if so how. Any suggestions from anyone would be greatly appreciated. I have been stuck on this for a couple of days and I really need to move on. I am dreading the thought of uninstalling\reinstalling SQL2000 and 2005 on my machine but will if I have to. Please help!!!!|||I uninstalled and reinstalled SQL 2005 and then attempted to run my package and it worked. There must be something that changed about the way it evaluates expressions in Derived Column tasks or maybe it is the expression language itself. Either way I am going to have to move on without SP1 until residual issues like these are dealt with. I am going to submit a bug to Microsoft about it. If anyone comes up with anything, please post a response.|||

I have been unable to reproduce this either on SQL Server 2005 RTM or SP1.

Could you perhaps hook up the error output of the derived column and redirect error rows, and then see what value(s) of LastWorkedDate it's failing on?

Thanks,
mark

|||I apologize for taking so long to get back to you. Unfortunately, I have already reinstalled SQL to RTM -SP1. I lost two days trying to troubleshoot the problem and had to give up for now. As soon as I have a chance I will reinstall SP1 and try again. For now however I have to live without it. Thank you very much for trying to reproduce the error. I will post more about this once I have the time.|||

DatabaseOgre wrote:

I saw a post where someone got the same error if they tried to use an expression language date function with a date value outside of SQL range (1/1/1753 to 12/31/9999). This post however was pre-SP1. I wonder whether that issue was addressed and if so how.

I suspect that what you suggested is indeed the problem. YEAR will fail if the date is outside of the SQL range.

If you have a chance to try redirecting rows to see if the values are indeed outside of that range, that would help us be sure that we have figured this one out. In the meantime, I will update the bug with this information.

|||

Sorry. I went back to the post and reade it a little slower this time. The post refers specifically to the DATEPART function with the YEAR argument. Here is a quote:

Can someone confirm this for me? The expression language in SSIS has the same limitations on date ranges as Sql Server? That limitation is that valid date ranges are from Jan 1, 1753 to Dec 31, 9999.

When ever I try to do a date function (DATEPART, for example) in a Derived Column Transformation on a date less than 1/1/1753, I get an error. I initially discovered this when bringing data over from Oracle to Sql Server. Just as a test, I created a text file filled with various dates and tried to import it. Whenever a date is less than 1/1/1753, it blows up.

For example, this expression code - DATEPART("YEAR",Date) will yield this error - [Derived Column [24]] Error: The "component "Derived Column" (24)" failed because error code 0xC0049067 occurred, and the error row disposition on "output column "YEAR" (80)" specifies failure on error. An error occurred on the specified object of the specified component.

As a workaround, I've been using a Script Component to do date checking, but this is obviously not ideal.

Jamie Thomson verified it for him and it was submitted as a bug for not being able to handle dates outside of the T-SQL date range. I checked my data to look for any dates that were outside of the range (pre-package) and did not find any. I also tried handling any such dates inside of a Script Task (as a just in case) which ran prior to the Derived Column Task and still received the error. When I get some time I will take a look at my source data again and see if there is anything there. I doubt it however because I am using the same package and same data source with SQL2005 RTM and I am not getting any errors. I probably wouldn't change much about the bug I submitted just yet.

|||

The value being out of range was my best guess for the cause of the error. If you know the values are all in range, then something else is wrong.

The best thing now, I think, would be for you to try redirecting the error rows and see what values are causing the problem. Once we know that, I will be able to try to reproduce the problem.

Thanks!
Mark

Derived column not processing any records

Hi,

I have built a package in which i use a derived column to create a new set of columns and then pass the same to another target transformation.

The issue now what I am facing is, the re are certain number of records coming from source(16 rows) and gets processed before the Derived Column transformation, but after that, no records gets processed after the derived column transformation.

The package status shows as Success, but there is no records being written in the target table.

Any idea what could be the issue here?

Thanks,

Manish

Have you used data viewers on the data flow to see why records are not "passing" through? The records HAVE to go through the derived column -- it can't stop them.

Are you positive the records are not coming out of the derived column transformation?|||

Are no records being written to your destination, or you're just missing the derived values?

If the latter, I'd also check your column mappings in your destination (or any transformations downstream of your Derived Column Trx and upstream of your destination) to make sure your new columns are mapped. This is particularly important if you added the Derived Column Trx later in the development of your data flow. The mappings won't automatically appear.

|||

Hi Phil,

Thanks for the response.

Yes, I have put data viewers before and after the derived column transformation. The data viewer before the transformation shows the 13 records being passed, but the data viewer after the transformation doesnt show up any record, it is blank. I have tried many option in the mapping, but no luck.

I am now re-creating the mapping to check if the issue persists.

THough, let me know if you can think of any other scenario which could help in resolving this.

Thanks,

Manish

|||

ManishSingh wrote:

Hi Phil,

Thanks for the response.

Yes, I have put data viewers before and after the derived column transformation. The data viewer before the transformation shows the 13 records being passed, but the data viewer after the transformation doesnt show up any record, it is blank. I have tried many option in the mapping, but no luck.

I am now re-creating the mapping to check if the issue persists.

THough, let me know if you can think of any other scenario which could help in resolving this.

Thanks,

Manish

What mapping? There are no mappings in the derived column transformation.|||By mapping I was referring to the package. I tried various options to delete transformation and add another etc in the package to check for the data flow|||

ManishSingh wrote:

By mapping I was referring to the package. I tried various options to delete transformation and add another etc in the package to check for the data flow

Make a copy of your package for this test.

Leave everything as is, but remove EVERYTHING AFTER the derived column transformation and add a row count transformation. Hook it up to the derived column transformation. This will be your new destination. You'll need to add a variable for the row counter, of course.

Run the package. What happens? Does the derived column pass all rows through now?|||

Phil,

Thanks for your response. I was out of office, so couldnt reply back.

I tried adding the row count as the destination after the derived column transformation, and it was running successfully. Then again, I removed the row count and added the transformation again, and it started working. I am not sure, what could be the issue, but now its running fine.

Thanks for all your help. Really appreciate it!!!!!!!!

sql

Derived column not processing any records

Hi,

I have built a package in which i use a derived column to create a new set of columns and then pass the same to another target transformation.

The issue now what I am facing is, the re are certain number of records coming from source(16 rows) and gets processed before the Derived Column transformation, but after that, no records gets processed after the derived column transformation.

The package status shows as Success, but there is no records being written in the target table.

Any idea what could be the issue here?

Thanks,

Manish

Have you used data viewers on the data flow to see why records are not "passing" through? The records HAVE to go through the derived column -- it can't stop them.

Are you positive the records are not coming out of the derived column transformation?|||

Are no records being written to your destination, or you're just missing the derived values?

If the latter, I'd also check your column mappings in your destination (or any transformations downstream of your Derived Column Trx and upstream of your destination) to make sure your new columns are mapped. This is particularly important if you added the Derived Column Trx later in the development of your data flow. The mappings won't automatically appear.

|||

Hi Phil,

Thanks for the response.

Yes, I have put data viewers before and after the derived column transformation. The data viewer before the transformation shows the 13 records being passed, but the data viewer after the transformation doesnt show up any record, it is blank. I have tried many option in the mapping, but no luck.

I am now re-creating the mapping to check if the issue persists.

THough, let me know if you can think of any other scenario which could help in resolving this.

Thanks,

Manish

|||

ManishSingh wrote:

Hi Phil,

Thanks for the response.

Yes, I have put data viewers before and after the derived column transformation. The data viewer before the transformation shows the 13 records being passed, but the data viewer after the transformation doesnt show up any record, it is blank. I have tried many option in the mapping, but no luck.

I am now re-creating the mapping to check if the issue persists.

THough, let me know if you can think of any other scenario which could help in resolving this.

Thanks,

Manish

What mapping? There are no mappings in the derived column transformation.|||By mapping I was referring to the package. I tried various options to delete transformation and add another etc in the package to check for the data flow|||

ManishSingh wrote:

By mapping I was referring to the package. I tried various options to delete transformation and add another etc in the package to check for the data flow

Make a copy of your package for this test.

Leave everything as is, but remove EVERYTHING AFTER the derived column transformation and add a row count transformation. Hook it up to the derived column transformation. This will be your new destination. You'll need to add a variable for the row counter, of course.

Run the package. What happens? Does the derived column pass all rows through now?|||

Phil,

Thanks for your response. I was out of office, so couldnt reply back.

I tried adding the row count as the destination after the derived column transformation, and it was running successfully. Then again, I removed the row count and added the transformation again, and it started working. I am not sure, what could be the issue, but now its running fine.

Thanks for all your help. Really appreciate it!!!!!!!!

Sunday, March 25, 2012

derived column help

Hi Guys,

How can i put in a derived column the value of 3 columns? I've tried these:

[MyId]+[Paper1]

"[MyId]+[Paper1]"

but no luck. How can i put it right?

Thanks

Gemma

The first one looks correct. [Column1] + [Column2] + [Column3]

Note: Are the three columns you are trying to combine all of the same type (i.e. unicode string, string, etc)? If not you will probably need to either convert them before trying this or cast them inside of the derived column transform.

|||

Maybe you have a type mismatch or a typo?

Try type casting and then concatenating. e.g. (DT_WSTR) [MyID] + [Paper1]

The top right box in the transformation editor contains these functions, so dont sweat the syntax, and the Double Inverted commas etc.

The top left box contains available variables and columns. Dragging and dropping your columns into the expression will avoid typos.

HTH

|||Any error messages?|||

Hi,

I have 3 different data type columns. One is int, 2nd one is varchar and 3rd one is datetime.

What can i do as still in the expression its coming up as red and i can't press ok.

Maybe i'm not putting in the function right.

Can someone tell me how to put them in the expression box?

Thanks

Gemma

|||

Hi Guys,

Here is the error message:

TITLE: Microsoft Visual Studio

Error at GetMyData [Derived Column [548]]: The data types "DT_I4" and "DT_WSTR" are incompatible for binary operator "+". The operand types could not be implicitly cast into compatible types for the operation. To perform this operation, one or both operands need to be explicitly cast with a cast operator.

Error at GetMyData [Derived Column [548]]: Attempt to set the result type of binary operation "MyID + MyNo" failed with error code 0xC0047080.

Error at GetMyData [Derived Column [548]]: Computing the expression "[MyID] + [MyNo] + [Paper1]" failed with error code 0xC0047084. The expression may have errors, such as divide by zero, that cannot be detected at parse time, or there may be an out-of-memory error.

Error at GetMyData [Derived Column [548]]: The expression "[MyID] + [MyNo] + [Paper1]" on "output column "MyNewID" (684)" is not valid.

Error at GetMyData [Derived Column [548]]: Failed to set property "Expression" on "output column "MyNewID" (684)".

ADDITIONAL INFORMATION:

Exception from HRESULT: 0xC0204006 (Microsoft.SqlServer.DTSPipelineWrap)

Gemma

|||

You must convert to string before concactenate.

Regards!!

|||

Hi,

If i knew how i would have done it by now

Can you tell me how should I with the syntax?

Thanks

Gemma

|||(DT_WSTR) [MyID] + (DT_WSTR)[Paper1]|||

David Frommer wrote:

(DT_WSTR) [MyID] + (DT_WSTR)[Paper1]

The above examples by David and karfast are close, but not quite right.

(DT_WSTR,length)[MyID] + (DT_WSTR,length)[Paper1] - (where length is the number of bytes long the string is to be.)

So, for example, you could have:
(DT_WSTR,50)[MyID] + (DT_WSTR,50)[Paper1]|||

OK, try:

(DT_WSTR,2)([Column1])+(DT_WSTR,2)([Column2])+(DT_WSTR,2)([Column3])

Replace 2 for the lenght of your columns...

I hope helped!!!

|||

Thanks pedro.

you're a life saver.

Gemma

derived column - lots of columns-automate?

I'm sending a lot of columns through my derived column transform, checking for empty strings from a flat file - I was wondering is there a way that I could "script out" all the transforms instead of enduring this click hell that I'm stuck in inside the derived column transformation editor. I've got probably 100+ columns to configure with the following sort of transform....

Replace Col1

TRIM(Col1) == "" ? NULL(DT_WSTR,2) : Col1

Basically - if the string is empty, then throw Null in the data stream. I'm about a third the way through but it would really be nice if there was a quicker way. Even with the most efficient copying & pasting & keyboard shortcuts, it's still painful.

Using a script component, you can loop through the columns and perform this operation. Here's a few samples:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=864401&SiteID=1

http://agilebi.com/cs/blogs/jwelch/archive/2007/06/01/xml-destination-script-component.aspx

The top one requires a little more coding, but is more flexible. The other uses reflection, which is slower but requires less code. Or you could use something like this, a slimmed down version of the first one that only handles strings.

Code Snippet

'Note: this code was originally written/posted by the SSIS forum user, jaegd. http://forums.microsoft.com/MSDN/User/Profile.aspx?UserID=133544&SiteID=1

'Credit has been given where credit is due

'Original post: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=864401&SiteID=1

'Trimmed to bare minimum code by jwelch

Imports Microsoft.SqlServer.Dts.Pipeline

Public Class ScriptMain

Inherits UserComponent

Private inputBuffer As PipelineBuffer

Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)

Dim counter As Integer = 0

For counter = 0 To inputBuffer.ColumnCount - 1

If (inputBuffer.GetString(counter)) = "." Then

inputBuffer.SetString(counter, Chr(0))

End If

Next

End Sub

Public Overrides Sub ProcessInput(ByVal InputID As Integer, ByVal Buffer As Microsoft.SqlServer.Dts.Pipeline.PipelineBuffer)

' Get the Pipeline Buffer for subsequent ordinal column access

inputBuffer = Buffer

MyBase.ProcessInput(InputID, Buffer)

End Sub

End Class

sql

Derived Column

How do i change the data type of a column using the derived column feature?
I have two columns that need changing
(a) needs to be changed to an integer
(b) needs to be changed to a date
Both columns contain balnks and spaces!
Realise i could do this using a view but is it best practice to change the
data types before they get into the Database
Milk Bottle> Realise i could do this using a view but is it best practice to change the
> data types before they get into the Database
Good idea. One of the main purposes of SSIS is to scrub data before it is
loaded.

> I have two columns that need changing
> (a) needs to be changed to an integer
> (b) needs to be changed to a date
One method is with 4 Derived Column transformations, 2 for the integer and 2
for the datetime. The first transformation of each pair attempts to convert
the input to the appropriate datatype and the second transformation assigns
a default value if a conversion error occurs. To do this:
Create 2 new derived column transforms for the integer field and specify
expression as (DT_I4)[YourIntegerData] in the first transformation. Con
nect
that derived column output path to a new Union All and the derived column
error path (redirect row) to the second transformation that specifies your
desired default value (e.g. (DT_I4)0). Connect that derived column output
path to the same union all as the first transformation so that all records
are processed and contain valid integer values.
Repeat the above process for the datetime field with the union all output of
the integer transforms connected to a derived column with expression
(DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union all
will
then contain only valid integer and datetime values.
Hope this helps.
Dan Guzman
SQL Server MVP
"MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
> How do i change the data type of a column using the derived column
> feature?
> I have two columns that need changing
> (a) needs to be changed to an integer
> (b) needs to be changed to a date
> Both columns contain balnks and spaces!
> Realise i could do this using a view but is it best practice to change the
> data types before they get into the Database
>
>
> --
> Milk Bottle|||Dan
Thank you very very much...it worked a treat.
Thanks for taking the time to post a reply.
Milk Bottle
"Dan Guzman" wrote:

> Good idea. One of the main purposes of SSIS is to scrub data before it is
> loaded.
>
> One method is with 4 Derived Column transformations, 2 for the integer and
2
> for the datetime. The first transformation of each pair attempts to conve
rt
> the input to the appropriate datatype and the second transformation assign
s
> a default value if a conversion error occurs. To do this:
> Create 2 new derived column transforms for the integer field and specify
> expression as (DT_I4)[YourIntegerData] in the first transformation. C
onnect
> that derived column output path to a new Union All and the derived column
> error path (redirect row) to the second transformation that specifies your
> desired default value (e.g. (DT_I4)0). Connect that derived column output
> path to the same union all as the first transformation so that all records
> are processed and contain valid integer values.
> Repeat the above process for the datetime field with the union all output
of
> the integer transforms connected to a derived column with expression
> (DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union al
l will
> then contain only valid integer and datetime values.
>
> --
> Hope this helps.
> Dan Guzman
> SQL Server MVP
> "MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
> news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
>|||I'm glad I was able to help you out.
Dan Guzman
SQL Server MVP
"MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
news:3FAD3449-D418-46FB-83CC-1ED98F1A6F1C@.microsoft.com...[vbcol=seagreen]
> Dan
> Thank you very very much...it worked a treat.
> Thanks for taking the time to post a reply.
> --
> Milk Bottle
>
> "Dan Guzman" wrote:
>

Derived Column

How do i change the data type of a column using the derived column feature?
I have two columns that need changing
(a) needs to be changed to an integer
(b) needs to be changed to a date
Both columns contain balnks and spaces!
Realise i could do this using a view but is it best practice to change the
data types before they get into the Database
--
Milk Bottle> Realise i could do this using a view but is it best practice to change the
> data types before they get into the Database
Good idea. One of the main purposes of SSIS is to scrub data before it is
loaded.
> I have two columns that need changing
> (a) needs to be changed to an integer
> (b) needs to be changed to a date
One method is with 4 Derived Column transformations, 2 for the integer and 2
for the datetime. The first transformation of each pair attempts to convert
the input to the appropriate datatype and the second transformation assigns
a default value if a conversion error occurs. To do this:
Create 2 new derived column transforms for the integer field and specify
expression as (DT_I4)[YourIntegerData] in the first transformation. Connect
that derived column output path to a new Union All and the derived column
error path (redirect row) to the second transformation that specifies your
desired default value (e.g. (DT_I4)0). Connect that derived column output
path to the same union all as the first transformation so that all records
are processed and contain valid integer values.
Repeat the above process for the datetime field with the union all output of
the integer transforms connected to a derived column with expression
(DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union all will
then contain only valid integer and datetime values.
Hope this helps.
Dan Guzman
SQL Server MVP
"MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
> How do i change the data type of a column using the derived column
> feature?
> I have two columns that need changing
> (a) needs to be changed to an integer
> (b) needs to be changed to a date
> Both columns contain balnks and spaces!
> Realise i could do this using a view but is it best practice to change the
> data types before they get into the Database
>
>
> --
> Milk Bottle|||Dan
Thank you very very much...it worked a treat.
Thanks for taking the time to post a reply.
--
Milk Bottle
"Dan Guzman" wrote:
> > Realise i could do this using a view but is it best practice to change the
> > data types before they get into the Database
> Good idea. One of the main purposes of SSIS is to scrub data before it is
> loaded.
> > I have two columns that need changing
> > (a) needs to be changed to an integer
> > (b) needs to be changed to a date
> One method is with 4 Derived Column transformations, 2 for the integer and 2
> for the datetime. The first transformation of each pair attempts to convert
> the input to the appropriate datatype and the second transformation assigns
> a default value if a conversion error occurs. To do this:
> Create 2 new derived column transforms for the integer field and specify
> expression as (DT_I4)[YourIntegerData] in the first transformation. Connect
> that derived column output path to a new Union All and the derived column
> error path (redirect row) to the second transformation that specifies your
> desired default value (e.g. (DT_I4)0). Connect that derived column output
> path to the same union all as the first transformation so that all records
> are processed and contain valid integer values.
> Repeat the above process for the datetime field with the union all output of
> the integer transforms connected to a derived column with expression
> (DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union all will
> then contain only valid integer and datetime values.
>
> --
> Hope this helps.
> Dan Guzman
> SQL Server MVP
> "MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
> news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
> >
> > How do i change the data type of a column using the derived column
> > feature?
> > I have two columns that need changing
> > (a) needs to be changed to an integer
> > (b) needs to be changed to a date
> > Both columns contain balnks and spaces!
> >
> > Realise i could do this using a view but is it best practice to change the
> > data types before they get into the Database
> >
> >
> >
> >
> >
> > --
> > Milk Bottle
>|||I'm glad I was able to help you out.
--
Dan Guzman
SQL Server MVP
"MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
news:3FAD3449-D418-46FB-83CC-1ED98F1A6F1C@.microsoft.com...
> Dan
> Thank you very very much...it worked a treat.
> Thanks for taking the time to post a reply.
> --
> Milk Bottle
>
> "Dan Guzman" wrote:
>> > Realise i could do this using a view but is it best practice to change
>> > the
>> > data types before they get into the Database
>> Good idea. One of the main purposes of SSIS is to scrub data before it
>> is
>> loaded.
>> > I have two columns that need changing
>> > (a) needs to be changed to an integer
>> > (b) needs to be changed to a date
>> One method is with 4 Derived Column transformations, 2 for the integer
>> and 2
>> for the datetime. The first transformation of each pair attempts to
>> convert
>> the input to the appropriate datatype and the second transformation
>> assigns
>> a default value if a conversion error occurs. To do this:
>> Create 2 new derived column transforms for the integer field and specify
>> expression as (DT_I4)[YourIntegerData] in the first transformation.
>> Connect
>> that derived column output path to a new Union All and the derived column
>> error path (redirect row) to the second transformation that specifies
>> your
>> desired default value (e.g. (DT_I4)0). Connect that derived column
>> output
>> path to the same union all as the first transformation so that all
>> records
>> are processed and contain valid integer values.
>> Repeat the above process for the datetime field with the union all output
>> of
>> the integer transforms connected to a derived column with expression
>> (DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union all
>> will
>> then contain only valid integer and datetime values.
>>
>> --
>> Hope this helps.
>> Dan Guzman
>> SQL Server MVP
>> "MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
>> news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
>> >
>> > How do i change the data type of a column using the derived column
>> > feature?
>> > I have two columns that need changing
>> > (a) needs to be changed to an integer
>> > (b) needs to be changed to a date
>> > Both columns contain balnks and spaces!
>> >
>> > Realise i could do this using a view but is it best practice to change
>> > the
>> > data types before they get into the Database
>> >
>> >
>> >
>> >
>> >
>> > --
>> > Milk Bottlesql

Derived Column

I have two columns made up of 4 digits numbers eg col1 1234, col2 9876

I want to create a derived column so i get a eight digit column, eg col3 12349876

I cannot seem to get the expression right, I seem to be always adding the value which i do not want, can someone help me out with the expression

Thanks

Robbie

if you just use the '+' operator, the values would get added. To fix that just cast the values to string before concatenating them.

something like:

(DT_STR,4,1252)col1 + (DT_STR,4,1252)col2

|||

Worked like a dream thanks,

One day I will understand all this !!!!

|||test|||test agauin|||test gain, this must be my last|||

Please mark the thread as answered...

thanks

|||

Note that if your 4-digit numbers could begin with one or more zeros, you would need to do a bit more to ensure the result is 8 digits. For example, 1234 and 0056 would result in 123456 using the above expression. If you actually want 12340056, you would need to do something like this:

RIGHT("0000" + (DT_WSTR,4)col1, 4)) + (RIGHT("0000" + (DT_WSTR,4)col2, 4)

Wrap that whole expression in a cast back to an integer type if that is what you need.

|||

Rafael Salas wrote:

Please mark the thread as answered...

thanks

This is done how?

|||

Mark Durley wrote:

Note that if your 4-digit numbers could begin with one or more zeros, you would need to do a bit more to ensure the result is 8 digits. For example, 1234 and 0056 would result in 123456 using the above expression. If you actually want 12340056, you would need to do something like this:

RIGHT("0000" + (DT_WSTR,4)col1, 4)) + (RIGHT("0000" + (DT_WSTR,4)col2, 4)

Wrap that whole expression in a cast back to an integer type if that is what you need.

Thanks mark this is great stuff, I have another problem but will create a new thread

|||

Personally, I would use

(col1 * 10000) + col2

- If you want to keep the leading zero's, that is.

Pipo1

|||

Pipo1 wrote:

Personally, I would use

(col1 * 10000) + col2

- If you want to keep the leading zero's, that is.

Pipo1

A much simpler (and more efficient) solution, especially if you want to end up with a number still :)

|||

Mark Durley wrote:

Pipo1 wrote:

Personally, I would use

(col1 * 10000) + col2

- If you want to keep the leading zero's, that is.

Pipo1

A much simpler (and more efficient) solution, especially if you want to end up with a number still :)

I may be wrong...but that expression will keep leading zeros for col2 only...what if you want to keep the leading zeros of col1?

|||

Rafael Salas wrote:

I may be wrong...but that expression will keep leading zeros for col2 only...what if you want to keep the leading zeros of col1?

You'll lose them. That's why I like using the right() function.|||

Phil Brammer wrote:

Rafael Salas wrote:

I may be wrong...but that expression will keep leading zeros for col2 only...what if you want to keep the leading zeros of col1?

You'll lose them. That's why I like using the right() function.

Thanks Phil for confirming that. It is good to know I am not crazy

Derived Column

How do i change the data type of a column using the derived column feature?
I have two columns that need changing
(a) needs to be changed to an integer
(b) needs to be changed to a date
Both columns contain balnks and spaces!
Realise i could do this using a view but is it best practice to change the
data types before they get into the Database
Milk Bottle
> Realise i could do this using a view but is it best practice to change the
> data types before they get into the Database
Good idea. One of the main purposes of SSIS is to scrub data before it is
loaded.

> I have two columns that need changing
> (a) needs to be changed to an integer
> (b) needs to be changed to a date
One method is with 4 Derived Column transformations, 2 for the integer and 2
for the datetime. The first transformation of each pair attempts to convert
the input to the appropriate datatype and the second transformation assigns
a default value if a conversion error occurs. To do this:
Create 2 new derived column transforms for the integer field and specify
expression as (DT_I4)[YourIntegerData] in the first transformation. Connect
that derived column output path to a new Union All and the derived column
error path (redirect row) to the second transformation that specifies your
desired default value (e.g. (DT_I4)0). Connect that derived column output
path to the same union all as the first transformation so that all records
are processed and contain valid integer values.
Repeat the above process for the datetime field with the union all output of
the integer transforms connected to a derived column with expression
(DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union all will
then contain only valid integer and datetime values.
Hope this helps.
Dan Guzman
SQL Server MVP
"MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
> How do i change the data type of a column using the derived column
> feature?
> I have two columns that need changing
> (a) needs to be changed to an integer
> (b) needs to be changed to a date
> Both columns contain balnks and spaces!
> Realise i could do this using a view but is it best practice to change the
> data types before they get into the Database
>
>
> --
> Milk Bottle
|||Dan
Thank you very very much...it worked a treat.
Thanks for taking the time to post a reply.
Milk Bottle
"Dan Guzman" wrote:

> Good idea. One of the main purposes of SSIS is to scrub data before it is
> loaded.
>
> One method is with 4 Derived Column transformations, 2 for the integer and 2
> for the datetime. The first transformation of each pair attempts to convert
> the input to the appropriate datatype and the second transformation assigns
> a default value if a conversion error occurs. To do this:
> Create 2 new derived column transforms for the integer field and specify
> expression as (DT_I4)[YourIntegerData] in the first transformation. Connect
> that derived column output path to a new Union All and the derived column
> error path (redirect row) to the second transformation that specifies your
> desired default value (e.g. (DT_I4)0). Connect that derived column output
> path to the same union all as the first transformation so that all records
> are processed and contain valid integer values.
> Repeat the above process for the datetime field with the union all output of
> the integer transforms connected to a derived column with expression
> (DT_DBTIMESTAMP)[YourDateTimeData]. The output of the second union all will
> then contain only valid integer and datetime values.
>
> --
> Hope this helps.
> Dan Guzman
> SQL Server MVP
> "MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
> news:488C7D26-B9A6-43F8-BB9D-596EF19B2517@.microsoft.com...
>
|||I'm glad I was able to help you out.
Dan Guzman
SQL Server MVP
"MilkBottle" <MilkBottle@.discussions.microsoft.com> wrote in message
news:3FAD3449-D418-46FB-83CC-1ED98F1A6F1C@.microsoft.com...[vbcol=seagreen]
> Dan
> Thank you very very much...it worked a treat.
> Thanks for taking the time to post a reply.
> --
> Milk Bottle
>
> "Dan Guzman" wrote:

Derive columns from cube member?

I have a cube that I read in a data flow section using OLE DB. To get this to work I had to do an ad hoc query through a sql server database ( this is problem that is described in post 219068).

So I now have the data and I want to derive new columns based on that data using an if statement:

if ( ISNULL(" [Facility].[REGION].[NATCODE].[MEMBER_CAPTION] " ) comptype = "NAT" else comptype = "REG"

comptype is the new derived column I am creating.

But when I try to save this I get an error: The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a requred element such as parenthesis

I selected the cube element from the columns list in derived element, when it initially put it in it looked like:

[[Facility].[REGION].[NATCODE].[MEMBER_CAPTION] ]

and gave error:

The token "[" was not recognized. The expression cannot be parsed because it contains invalid elements at the location specified.

Below is the statement I use to get data back from AS

select * from openrowset('MSOLAP', 'DATASOURCE=local; Initial Catalog=Patient Demographics 2005;',
'with member [Measures].[TimeDisplayName] as [Calendar].CurrentMember.Name
SELECT NON EMPTY { [Measures].[TimeDisplayName],[Measures].[Case Count],
[Measures].[A100 Bathing],[Measures].[A100 Bed Chair Wheelchair],
[Measures].[A100 Bladder],[Measures].[A100 Bowel],
[Measures].[A100 Dressing Lower],[Measures].[A100 Dressing Upper],
[Measures].[A100 Eating],[Measures].[A100 Grooming],
[Measures].[A100 Stairs],[Measures].[A100 Toilet],
[Measures].[A100 Toileting],[Measures].[A100 Tub Shower],
[Measures].[A100 Walk Wheelchair],[Measures].[A200 Comprehension],
[Measures].[A200 Expression],[Measures].[A200 Interaction],
[Measures].[A200 Memory],[Measures].[A200 Problem Solving],
[Measures].[D100 Bathing],[Measures].[D100 Bed Chair Wheelchair],
[Measures].[D100 Bladder],[Measures].[D100 Bowel],
[Measures].[D100 Dressing Lower],[Measures].[D100 Dressing Upper],
[Measures].[D100 Eating],[Measures].[D100 Grooming],
[Measures].[D100 Stairs],[Measures].[D100 Toilet],
[Measures].[D100 Toileting],[Measures].[D100 Tub Shower],
[Measures].[D100 Walk Wheelchair],[Measures].[D200 Comprehension],
[Measures].[D200 Expression],[Measures].[D200 Interaction],
[Measures].[D200 Memory],[Measures].[D200 Problem Solving] } ON COLUMNS,
({[Facility].[REGION].[NATCODE], [Facility].[REGION].[REGCODE]} *
[RIC].[CMGGRPCD].ALLMEMBERS ) ON ROWS FROM [CMG Demographics]
WHERE [Calendar].[Quarter 2 (2007)]')

So can some one help me on how I can do this derived colum.

Try this...

isnull(<fieldname>) == true ? "NAT" : "REG"

To get <fieldname> expand the 'columns' folder in the derived column transformation editor.

|||Use a derived column to create new columns.

New Column Name: comptype
Expression: ISNULL([InputColumn]) ? "NAT" : "REG"|||

Anthony Martin wrote:

Try this...

isnull(<fieldname>) == true ? "NAT" : "REG"

Anthony, just simplify that statement. ISNULL() returns a boolean value, so there's no need to doubly check for a true status or not. Just drop the "== true" part.|||

Thanks, good point. Just a habit I guess.

|||

I have figured out what I had to do, but I do not know why I had to do this way, but first while I saw after I posted that used used bad assignment of derived column comptype = <value>. that was one of my many different attempts around the question I posted about.

1st) I could never get the if else statment to work in derived field, I had to use conditional if (?Smile logic for my ifs

2nd) I could not referenct the the MDX columns in my conditional if. It did not like all of the '[' and ']', so I had to convert all of the columns from MDX column name to a more standard sql column name, basically renamed column to not have '[', ']' in them

Does any one know why this does not work neatly, MDX is supposed to have been integrated with SSIS, but I had to do multiple tricks to get the data, right from start where I had to use ad hoc query though a sql database to execute the MDX.

This also results in fact that during selection of MDX it did not maintain any of the data types and converted everything to chararacter field of size 4000, which gave me trunctaion warnings in the selection of MDX into SSIS

Derive columns from cube member?

I have a cube that I read in a data flow section using OLE DB. To get this to work I had to do an ad hoc query through a sql server database ( this is problem that is described in post 219068).

So I now have the data and I want to derive new columns based on that data using an if statement:

if ( ISNULL(" [Facility].[REGION].[NATCODE].[MEMBER_CAPTION] " ) comptype = "NAT" else comptype = "REG"

comptype is the new derived column I am creating.

But when I try to save this I get an error: The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a requred element such as parenthesis

I selected the cube element from the columns list in derived element, when it initially put it in it looked like:

[[Facility].[REGION].[NATCODE].[MEMBER_CAPTION] ]

and gave error:

The token "[" was not recognized. The expression cannot be parsed because it contains invalid elements at the location specified.

Below is the statement I use to get data back from AS

select * from openrowset('MSOLAP', 'DATASOURCE=local; Initial Catalog=Patient Demographics 2005;',
'with member [Measures].[TimeDisplayName] as [Calendar].CurrentMember.Name
SELECT NON EMPTY { [Measures].[TimeDisplayName],[Measures].[Case Count],
[Measures].[A100 Bathing],[Measures].[A100 Bed Chair Wheelchair],
[Measures].[A100 Bladder],[Measures].[A100 Bowel],
[Measures].[A100 Dressing Lower],[Measures].[A100 Dressing Upper],
[Measures].[A100 Eating],[Measures].[A100 Grooming],
[Measures].[A100 Stairs],[Measures].[A100 Toilet],
[Measures].[A100 Toileting],[Measures].[A100 Tub Shower],
[Measures].[A100 Walk Wheelchair],[Measures].[A200 Comprehension],
[Measures].[A200 Expression],[Measures].[A200 Interaction],
[Measures].[A200 Memory],[Measures].[A200 Problem Solving],
[Measures].[D100 Bathing],[Measures].[D100 Bed Chair Wheelchair],
[Measures].[D100 Bladder],[Measures].[D100 Bowel],
[Measures].[D100 Dressing Lower],[Measures].[D100 Dressing Upper],
[Measures].[D100 Eating],[Measures].[D100 Grooming],
[Measures].[D100 Stairs],[Measures].[D100 Toilet],
[Measures].[D100 Toileting],[Measures].[D100 Tub Shower],
[Measures].[D100 Walk Wheelchair],[Measures].[D200 Comprehension],
[Measures].[D200 Expression],[Measures].[D200 Interaction],
[Measures].[D200 Memory],[Measures].[D200 Problem Solving] } ON COLUMNS,
({[Facility].[REGION].[NATCODE], [Facility].[REGION].[REGCODE]} *
[RIC].[CMGGRPCD].ALLMEMBERS ) ON ROWS FROM [CMG Demographics]
WHERE [Calendar].[Quarter 2 (2007)]')

So can some one help me on how I can do this derived colum.

Try this...

isnull(<fieldname>) == true ? "NAT" : "REG"

To get <fieldname> expand the 'columns' folder in the derived column transformation editor.

|||Use a derived column to create new columns.

New Column Name: comptype
Expression: ISNULL([InputColumn]) ? "NAT" : "REG"|||

Anthony Martin wrote:

Try this...

isnull(<fieldname>) == true ? "NAT" : "REG"

Anthony, just simplify that statement. ISNULL() returns a boolean value, so there's no need to doubly check for a true status or not. Just drop the "== true" part.|||

Thanks, good point. Just a habit I guess.

|||

I have figured out what I had to do, but I do not know why I had to do this way, but first while I saw after I posted that used used bad assignment of derived column comptype = <value>. that was one of my many different attempts around the question I posted about.

1st) I could never get the if else statment to work in derived field, I had to use conditional if (?Smile logic for my ifs

2nd) I could not referenct the the MDX columns in my conditional if. It did not like all of the '[' and ']', so I had to convert all of the columns from MDX column name to a more standard sql column name, basically renamed column to not have '[', ']' in them

Does any one know why this does not work neatly, MDX is supposed to have been integrated with SSIS, but I had to do multiple tricks to get the data, right from start where I had to use ad hoc query though a sql database to execute the MDX.

This also results in fact that during selection of MDX it did not maintain any of the data types and converted everything to chararacter field of size 4000, which gave me trunctaion warnings in the selection of MDX into SSIS

Friday, February 17, 2012

Dependent columns

I have three columns (CWOC, C, and PWC). both CWOC and PWC are dependent on the value in C, yet C is dependent on the value in PWC.

Basically:

C = PWC * BR
PWC = MR - CWOC
CWOC = MC - C

In setting up these columns in the database how do I set up columns like these that are so dependent on each other?

This was set up previously in Excel using iteration, but now I'm recreating the application in VB.net and SQL Server.

Thanks for your suggestions.There is no single logical solution to that set of equations. There are, in fact, infinite stable solutions. So how are you going to select one? And what good is it? Is this some sort of fractal database or chaos theory experiment?|||That's just how the client calculates their commissions.|||Oh.

My.

God.

I once had the task of setting up a commision and compensation system for territory sales reps. The Regional Director wanted something simple and understandable. So I gave them something simple and understandable. Then they wanted it to consider existing market saturation in each territory, so I threw in weighted averaging. Then they wanted it to account for seasonal variations, so I added an exponential smoothing model to the algorithm. Then they wanted it to account for historical performance, so I included a linear regression model in the algorithm.
You guessed it. None of the sales reps would accept the compensation model because it was too complicated. They would rather have something that was less than fully fair which they could calculate on their own, and I understood their point.

So how in the world did they end up with a non-linear commision structure?

I'm not just asking to be nosy, or out of incredulousness (ok, a bit of that...). But if you explain a bit more about the algorithm MAYBE I can help you come up with something...|||I give you my 2 cents opinion.

I put everything is under the hat of "business logic" in a stored procedure (with the aid of UDF if needed) or into an intermediate COM+ component. In this case, since the logic must be applied to the data contained in the database itself, i'd prefer the stored procedure solution.
This won't dirty your database design with business logic implication, so that in the future, if the business logic change it won't be necessary to change the db schema and vice-versa.

Dependencies

In Enterprise Manager (EM), how does it arrive at
the dependencies for a table?
The reason I am asking is, before I add additional
columns to a table, under "objects dependent on
<tablename>", it shows tables, procedures, triggers, etc.
After I add columns to this table, it shows only tables
but nothing else.
What could be the reason? What I may be doing wrong?
How can I relink everything?
Thank you,
KellyTo learn what EM does, run a profiler trace. You will probably find that EM calls a number of stored
procedures. So, you can read the source code for those stored procedures to determine whether you should
consider this a bug and possibly open a case with MS Support.
--
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"Kelly" <anonymous@.discussions.microsoft.com> wrote in message news:1cb6901c4532d$bcccab20$a301280a@.phx.gbl...
> In Enterprise Manager (EM), how does it arrive at
> the dependencies for a table?
> The reason I am asking is, before I add additional
> columns to a table, under "objects dependent on
> <tablename>", it shows tables, procedures, triggers, etc.
> After I add columns to this table, it shows only tables
> but nothing else.
> What could be the reason? What I may be doing wrong?
> How can I relink everything?
> Thank you,
> Kelly|||Depending on the nature of the table changes EM may create a new table, copy the data, and drop the original table. This process causes the dependency records to be lost. The dependency records can be restored by dropping and creating all the dependent objects in the proper sequence, or by...
Two of the SQL Server Tools applications have an option to restore dependency records automatically. WT2 and WT3 implement that feature in the same way, but then they diverge into different functionality. You can learn more at this URL...
http://www.sqlservertools.us
"Kelly" wrote:
> In Enterprise Manager (EM), how does it arrive at
> the dependencies for a table?
> The reason I am asking is, before I add additional
> columns to a table, under "objects dependent on
> <tablename>", it shows tables, procedures, triggers, etc.
> After I add columns to this table, it shows only tables
> but nothing else.
> What could be the reason? What I may be doing wrong?
> How can I relink everything?
> Thank you,
> Kelly
>|||Tibor,
Thanks for your input.
-Kelly
>--Original Message--
>To learn what EM does, run a profiler trace. You will
probably find that EM calls a number of stored
>procedures. So, you can read the source code for those
stored procedures to determine whether you should
>consider this a bug and possibly open a case with MS
Support.
>--
>Tibor Karaszi, SQL Server MVP
>http://www.karaszi.com/sqlserver/default.asp
>http://www.solidqualitylearning.com/
>
>"Kelly" <anonymous@.discussions.microsoft.com> wrote in
message news:1cb6901c4532d$bcccab20$a301280a@.phx.gbl...
>> In Enterprise Manager (EM), how does it arrive at
>> the dependencies for a table?
>> The reason I am asking is, before I add additional
>> columns to a table, under "objects dependent on
>> <tablename>", it shows tables, procedures, triggers,
etc.
>> After I add columns to this table, it shows only tables
>> but nothing else.
>> What could be the reason? What I may be doing wrong?
>> How can I relink everything?
>> Thank you,
>> Kelly
>
>.
>|||Thanks for the information.
-Kelly
>--Original Message--
>Depending on the nature of the table changes EM may
create a new table, copy the data, and drop the original
table. This process causes the dependency records to be
lost. The dependency records can be restored by dropping
and creating all the dependent objects in the proper
sequence, or by...
>Two of the SQL Server Tools applications have an option
to restore dependency records automatically. WT2 and WT3
implement that feature in the same way, but then they
diverge into different functionality. You can learn more
at this URL...
>http://www.sqlservertools.us
>"Kelly" wrote:
>> In Enterprise Manager (EM), how does it arrive at
>> the dependencies for a table?
>> The reason I am asking is, before I add additional
>> columns to a table, under "objects dependent on
>> <tablename>", it shows tables, procedures, triggers,
etc.
>> After I add columns to this table, it shows only tables
>> but nothing else.
>> What could be the reason? What I may be doing wrong?
>> How can I relink everything?
>> Thank you,
>> Kelly
>.
>

Dependant IDENTITY columns

Anyone know if MS-SQL Server supports IDENTITY columns that are incremented for each new value of the column it depends on.

For exameple:
Let's say I have a client table with a ClientID column as it's PRIMARY KEY.
This column can be an auto-incrementing IDENTITY column.

Then I have an orders table. The PRIMARY KEY for the orders table is composed of (ClientID, OrderID). I would like the OrderID to be an IDENTITY field that increments by an arbitrary value (1 in this case) for every new value of ClientID...therefore creating a unique PRIMARY KEY.

The contents of the table would look like this

ClientID OrderID
--- ---
1 0
1 1
2 0
1 2
2 1
2 2
1 3

and so on...

MySQL (and maybe other RDMS's...I haven't checked) seems to do this automatically when you set a column as AUTOINCREMENT and then define a composite PRIMARY KEY on two fields.
I know this can be done manually using triggers, but I was wondering if there was a better way...

Thanks in advanceYou are only allowed 1 IDENTITY Column per table...

How do you plan to INSERT the data in to the table?|||MySQL will do that? I'm skeptical...

Yes, you can do this using triggers, but it seems as if you are going to be using these values and their order as part of your application logic. That's generally not a good idea. An autoincrementing ID is a surrogate key, and should not have any inherent relationship to the data it represents.

What are you planning to use these values for?|||I know.

In an INSERT into the Orders table, I would supply the ClientID (and all the other fields I didn't mention in the orginal post) and the OrderID "for that client" would be incremented. Maybe you thought I meant that in the orders table the ClientID and OrderID are IDENTITY columns...|||BlindMan:
I wasn't planning on using it for anything (for now anyways), I was just curious. The Clients/Orders table was just a commonly used example.

As for it being possible with MySQL, I checked it again (since I first discovered this a while back), and yes it is possible. The syntax under MySQL would roughly be:

CREATE TABLE Clients (
ClientID INT AUTO_INCREMENT PRIMARY KEY);

CREATE TABLE Orders (
ClientID INT,
OrderID INT AUTO_INCREMENT,
CONSTRAINT PK_SomeName PRIMARY KEY (ClientID, OrderID));

Obviously there would be a foreign key set on Orders.ClientID...but I just want to show the example..

After populating Clients with a few values, you can insert a few into orders
like so:

INSERT INTO Orders (ClientID) VALUES (SomeValues);

and the OrderID field will increment like I mentionned in the original post,
except that the default AUTO_INCREMENT seed is "1", not "0".

Like I said, I was just curious if it was possible to use this with MS-SQL
without using a workaround.

Thanks for the information!!|||MySQL will do that? I'm skeptical...

oh ye of little faith, yes, it certainly can (as afx2029 so nicely illustrated)

mysql can do several things that make sense when you see them

other databases could learn from some of what they're doing

and before y'all jump all over me, i did not say mysql was better than other databases, just that they actually do some neat things

i'm still undecided about whether i like the ability to store 2004-05-00 and 2002-00-00 as perfectly valid datetime values (there's a good reason for it), but i am aghast at them allowing 2003-02-29 and 2001-02-31|||Rudy, what would be a good reason for allowing (2002-00-00) or (2004-05-00 ) as valid datetime?|||Yeah Rudy! If you love MySQL so much, why don't you just MARRY it? Huh? Well, why dontcha? If it's so much BETTER? :p|||GDMI, to allow you to use a single date column to record an event (birthday, battle, etc.) and be able to store partial information, e.g. if all you know about great-great-grand-uncle Fritz is that he was born in 1903 but you don't know the month or year, put 1903-00-00

you can't do that with a "normal" date column in other databases, which require an exact date, so you'd either have to carry separate year, month, day columns and allow nulls, or else fake it by putting 1903-01-01 (which, i hasten to point out, is wrong and misleading)

blindman, i've sworn off marriage, having gone through two of them

:cool:|||Rudy, Thanks for the explanation. I admit I haven't thought about that before ...|||Rudy, Thanks for the explanation. I admit I haven't thought about that before ...

I can't tell. Is he talking about your code or your marriages? I quit after one, so I guess that makes me a more efficient marriager than you. ;)|||I can't tell. Is he talking about your code or your marriages? I quit after one, so I guess that makes me a more efficient marriager than you. ;)Everybody should do it at least once. It helps promote the species, and it instills a real fear of the potential of hell.

This coming from a guy who's never found an eligible woman smart enough to be interesting and dumb enough to say "yes".

-PatP|||hell hath no fury like a woman with a divorce lawyer

but it is worth the pain

i have two fantastic boys from my first marriage, 32, and 30

and a fabulous daughter, 13, and son, 11, from my second

i would love to have even more kids, but i am broke, out of work, and too old and unattractive to have any hope of a third marriage

my only hope now is that one of my older boys has grandkids soon|||I can't tell. Is he talking about your code or your marriages?

Nope! I was talking about his code! ;)

But speaking about marriages .. I am preparing for my marriage .. hopefully it will be my first and last eventhough statistics are against that.

Blindman, Rudy you guys are great SQL consultants ;) in a "SQL World" that alone should qualify to get chicks! :p|||Careful, GDMI. Statistics shouw that just over half of the marriages in the US end in divorce. But just under half end in death!|||Nope! I was talking about his code! ;)

But speaking about marriages .. I am preparing for my marriage .. hopefully it will be my first and last eventhough statistics are against that.

Blindman, Rudy you guys are great SQL consultants ;) in a "SQL World" that alone should qualify to get chicks! :p

What IT offices have you been working in?|||a long time ago, i knew a woman who knew sql...

$expletive, was she ever hot!!!!!!!!!!

in my experience, IT has as many attractive women as any other vocation|||hell hath no fury like a woman with a divorce lawyer

but it is worth the pain

i have two fantastic boys from my first marriage, 32, and 30

and a fabulous daughter, 13, and son, 11, from my second

i would love to have even more kids, but i am broke, out of work, and too old and unattractive to have any hope of a third marriage

my only hope now is that one of my older boys has grandkids soonGood kids do help to ease the pain!

I can't comment on the "broke" part, and I don't see you as "unemployed" although I know that you'd debate that point. I'm not qualified to comment on unattractive (I only pay attention to females on that point), but somehow I still can't buy that one either.

Grandkids are great fun! They are even more fun than the first time around, and at least in my opinion give you your first real chance to "check your work"

-PatP|||I don't mean to be off-topic or anything....but is what I originally posted
possible with MS-SQL Server without resorting to triggers or another workaround?

:)|||Anyone know if MS-SQL Server supports IDENTITY columns that are incremented for each new value of the column it depends on.

For exameple:
Let's say I have a client table with a ClientID column as it's PRIMARY KEY.
This column can be an auto-incrementing IDENTITY column.

Then I have an orders table. The PRIMARY KEY for the orders table is composed of (ClientID, OrderID). I would like the OrderID to be an IDENTITY field that increments by an arbitrary value (1 in this case) for every new value of ClientID...therefore creating a unique PRIMARY KEY.

The contents of the table would look like this

ClientID OrderID
--- ---
1 0
1 1
2 0
1 2
2 1
2 2
1 3

and so on...

MySQL (and maybe other RDMS's...I haven't checked) seems to do this automatically when you set a column as AUTOINCREMENT and then define a composite PRIMARY KEY on two fields.
I know this can be done manually using triggers, but I was wondering if there was a better way...

Thanks in advance

No

I asked you how you plan to do the INSERT.

If you tell us, we might be able to suggest something...|||I don't mean to be off-topic or anything....but is what I originally posted
possible with MS-SQL Server without resorting to triggers or another workaround?

:)As Brett answered (twice), no. No truely relational database can support what you are looking for, since it is application specific and violates the single domain rule of first normal form as expressed in relational algebra.

The only way to add application specific code to SQL Server is via a trigger.

-PatP|||No truely relational database can support what you are looking for ...mysql can do it

or is that why you were so careful to qualify your statement? ;)

...since it is application specific and violates the single domain rule of first normal form as expressed in relational algebra.stop that! go to your room!

it's a feature

;)|||Just like

SELECT + 'This is a feature?'

Is a feature|||stop that! go to your room!

it's a feature

;)I just said that it wasn't relational, and pointed out how the feature could be simulated in a relational database. Was I naughty?

Just like

SELECT + 'This is a feature?'

Is a featureI see it that way!

-PatP