when etl process integrates data from various heterogeneous information sources, does metadata from these heterogeneous information sources go thru the etl process too?

mike.

I've had ETL tools build metadata for me before. I think there might have been a time (once) that I didn't regret it.

Although some tools will attempt to write metadata for me, I insist on reviewing it, and usually choose to re-write it manually. There might come a day when I'll trust an ETL tool to do this for me, but they've got a long way to go before I get there!

Metadata controls what the ETL tool can do for you. In general, ETL tools deal with MASSIVE amounts of data. I usually don't have enough spare disk/cpu/bandwidth to squander any of it (but then again, I'm a speed junkie too).

Your milage may vary, but I'll still roll my own metadata, thanks!

-PatP

Pat, are there any mechanisms or security features to allow metadata which has been placed in the repository to be limited to authorised persons only?

In some repositories yes, but in most it is assumed that if you have design (write) access to the repository, you have access to the MetaData. You'll have to check the documentation for your particular product/release.

-PatP

Metadata which is atteched to the source data, is it binded/integrated within the source data or separate ie separate schema? Also when the source data is extracted is it quite possible, say if the data was ie flat file or some other disparate type, the metadata may not be reccognised?

Different warehouses store metadata differently. In general, the metadata is physically stored in the warehouse, and logically bound to the actual data source or target. The source system is often ignorant of the metadata, but the warehouse still knows about it.

-PatP

so when data is imported into the DW from data sources the safest proceedure is to enter the metadata into the metadata repository in a passive fashion (ie hand written)?

does this way also resolve the issue of disparate metadata?

also what probs are also ripe when importing and integrating metadata schemas from data extracted from heterogeneous sources?

Hi pat, I think im confused with the metadata schema element.

Basically data which is to be integrated ie operational data, does this datas metadata get brought into the warehouse or is new metadata generated about this data?

For some reason i thought that when source data was imported into the warehouse, i thought its metadata schema was imported also? now im not sure?

can u explain what a metadata schema is, where it is generated, where it is stored and explain problems attached.


Mikey

The problem is that different warehouse/repository engines all handle metadata managment differently. Which warehouse/repository tools are you using?

-PatP

Also what are the main metadata standards orgs besides OMG?

and why is there not a single metadata standard?