¶ Data Product Creation Workflow
¶ Prerequisites
- User account with CREATE_DATA_PRODUCTS permission
- Assigned to a domain
- Understanding of data source
¶ Step-by-Step Creation Process
¶ Step 1: Meta (Basic Information)
-
Navigate to Data Products
- Click "Data Products" → "Create New"
-
Fill Basic Information:
Name: Customer Transaction History Key: customer-transaction-history (auto-generated) Description: Complete history of customer transactions including purchases, returns, and refunds Domain: Finance Classification: INTERNAL Update Frequency: Daily -
Select Transformer:
- None: Raw data, no transformation
- SQL: SQL-based transformations
-
Add Owner & Stewards:
Data Owners: [finance-team@company.com] Data Stewards: [data-engineering@company.com] -
Click "Next" to proceed to schema definition
¶ Step 2: Schema Method
Choose how to define your product schema:
Option 1: Manual Creation
- Type schema manually
- Best for simple schemas
- Full control over structure
Option 2: JSON/CSV Upload
- Upload sample file
- Schema extracted automatically
- Quick for existing files
Option 3: Database Connector
- Connect to database
- Import table schema
- Real-time sync available
Option 4: Staging Connector
- Use Elementrix staging area
- For unsupported sources
- Custom ingestion logic
Select your method and click "Next"
¶ Step 3: Manual / Upload / DB Connector / Staging Connector
(Depends on chosen method)
If Manual:
- Click "Add Field"
- Define each field:
Field Name: transaction_id Data Type: UUID Required: Yes Description: Unique transaction identifier
If Upload:
- Upload CSV or JSON file
- Review detected schema
- Adjust field types if needed
- Confirm schema
If Database Connector:
- Select database type (PostgreSQL/MySQL)
- Enter connection details:
Host: db.company.com Port: 5432 Database: transactions_db Username: readonly_user Password: •••••••• - Test connection
- Select table/view
- Import schema
If Staging Connector:
- Define staging area path
- Specify file format
- Define ingestion schedule
- Configure data validation
¶ Step 4: Fields Schema
Configure individual fields and quality attributes:
For Each Field:
-
Basic Properties:
Field Name: transaction_amount Display Name: Transaction Amount Data Type: Decimal(10,2) Required: Yes -
Quality Attributes:
☑ Unique: No ☑ Nullable: No ☑ PII: No ☑ Sensitive: No Validation Rules: - Minimum: 0.00 - Maximum: 1000000.00 - Format: ##,###.00 -
Business Context:
Description: Total transaction value in USD Business Rules: Includes taxes and fees Default Value: 0.00 Example Values: 49.99, 1299.00, 15.50 -
Data Quality Checks:
☑ Check for null values ☑ Validate range (0 - 1M) ☑ Format validation ☐ Custom validation rule
Repeat for all fields, then click "Next"
¶ Step 5: Workflow
Configure approval workflow for access requests:
Default Workflow:
Request → Owner Approval → Access Granted
Configure Approvers:
Step 1: Data Steward Review
Approvers: [data-steward@company.com]
Step 2: Owner Approval
Approvers: [owner@company.com]
Step 3: Governance Review (if sensitive)
Approvers: [governance@company.com]
Click "Next" when configured
¶ Step 6: Semantic Registry
Link business glossary terms to fields:
-
Search for Terms:
Search: "transaction" Results: - Transaction ID - Transaction Amount - Transaction Type - Transaction Date -
Map Terms to Fields:
Field: transaction_id Linked Term: Transaction ID Field: transaction_amount Linked Term: Transaction Amount Field: transaction_date Linked Term: Transaction Timestamp -
Add Product-Level Terms:
Data Product-Level Terms: - Customer Transaction - Financial Data - Transaction History -
Review Mappings:
- Ensure all important fields mapped
- Verify term definitions match usage
- Add new terms if needed
Click "Next" to continue
¶ Step 7: Edit Public Page
Create marketing content for data product discovery:
Marketing Page Sections:
-
Overview:
# Customer Transaction History Complete transactional data for all customer purchases, returns, and refunds. **Use Cases**: - Revenue analysis - Customer behavior insights - Fraud detection - Chargeback management -
Key Features:
✓ Real-time transaction data ✓ 7-year historical archive ✓ 99.9% uptime SLA ✓ Sub-second query response ✓ Daily data refresh -
Sample Queries:
-- Get transactions for customer SELECT * FROM transactions WHERE customer_id = '123e4567-e89b-12d3-a456-426614174000' ORDER BY transaction_date DESC; -- Calculate daily revenue SELECT DATE(transaction_date) as date, SUM(transaction_amount) as total_revenue FROM transactions WHERE transaction_date >= CURRENT_DATE - 30 GROUP BY DATE(transaction_date); -
Data Quality Metrics:
Completeness: 99.8% Accuracy: 99.95% Timeliness: < 5 minutes lag Consistency: 100% -
Support Information:
Contact: data-support@company.com Documentation: https://docs.company.com/transactions Slack Channel: #data-transactions
Click "Next" when complete
Result:
✓ Data Product Created Successfully!
Status: DRAFT
Next Steps:
1. Review product details
2. Test API endpoint
3. Submit for review when ready
¶ Next Steps
- Submit for Governance Approval
- Understand the Data Product Lifecycle
- Learn about Sync Strategies for different data sources