Results 1 to 11 of 11
  1. #1
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7

    Question on metadata for DataWarehouse

    when etl process integrates data from various heterogeneous information sources, does metadata from these heterogeneous information sources go thru the etl process too?

    mike.

  2. #2
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    I've had ETL tools build metadata for me before. I think there might have been a time (once) that I didn't regret it.

    Although some tools will attempt to write metadata for me, I insist on reviewing it, and usually choose to re-write it manually. There might come a day when I'll trust an ETL tool to do this for me, but they've got a long way to go before I get there!

    Metadata controls what the ETL tool can do for you. In general, ETL tools deal with MASSIVE amounts of data. I usually don't have enough spare disk/cpu/bandwidth to squander any of it (but then again, I'm a speed junkie too).

    Your milage may vary, but I'll still roll my own metadata, thanks!

    -PatP

  3. #3
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7
    Pat, are there any mechanisms or security features to allow metadata which has been placed in the repository to be limited to authorised persons only?

  4. #4
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    In some repositories yes, but in most it is assumed that if you have design (write) access to the repository, you have access to the MetaData. You'll have to check the documentation for your particular product/release.

    -PatP

  5. #5
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7
    Metadata which is atteched to the source data, is it binded/integrated within the source data or separate ie separate schema? Also when the source data is extracted is it quite possible, say if the data was ie flat file or some other disparate type, the metadata may not be reccognised?

  6. #6
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Different warehouses store metadata differently. In general, the metadata is physically stored in the warehouse, and logically bound to the actual data source or target. The source system is often ignorant of the metadata, but the warehouse still knows about it.

    -PatP

  7. #7
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7
    so when data is imported into the DW from data sources the safest proceedure is to enter the metadata into the metadata repository in a passive fashion (ie hand written)?

    does this way also resolve the issue of disparate metadata?

  8. #8
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7
    also what probs are also ripe when importing and integrating metadata schemas from data extracted from heterogeneous sources?

  9. #9
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7
    Hi pat, I think im confused with the metadata schema element.

    Basically data which is to be integrated ie operational data, does this datas metadata get brought into the warehouse or is new metadata generated about this data?

    For some reason i thought that when source data was imported into the warehouse, i thought its metadata schema was imported also? now im not sure?

    can u explain what a metadata schema is, where it is generated, where it is stored and explain problems attached.


    Mikey

  10. #10
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    The problem is that different warehouse/repository engines all handle metadata managment differently. Which warehouse/repository tools are you using?

    -PatP

  11. #11
    Join Date
    Mar 2004
    Location
    Sydney
    Posts
    7
    Also what are the main metadata standards orgs besides OMG?

    and why is there not a single metadata standard?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •