(23-05-25) - AWS, IBM Workshop

---

Created: =dateformat(this.file.ctime,"dd MMM yyyy, hh:mm a") | Modified: =dateformat(this.file.mtime,"dd MMM yyyy, hh:mm a") Tags: knowledge

---

Setup:

TeamRole:~/environment/ibm-aws-quickstart-immersionday-main/scripts $ ./create_users.sh
# initial_admin_password
Users created successfully.
Password for both user21756 and datascientist21756 is: password
providing access to service instances
 
 
TeamRole:~/environment/ibm-aws-quickstart-immersionday-main/scripts $ oc get route -n zen-46 |awk 'NR==2 {print $2}'
cpd-zen-46.apps.mod-a06dd33aea0643bb.cpd3qlr3x.ibmworkshops.com
 

IBM Cloud Pak for Data (CP4D) on AWS Modernization Workshop :: Immersion Day Workshop IBM Cloud Pak for Data (CP4D) on AWS Modernization Workshop :: English

IBM Data and AI Singapore Team

data governance

• on prem vs on cloud data sources
• challenges
  • distributed storage, ownership
  • regulatory req, sensitivity
  • collab
  • moving data

IBM + AWS

• collaboration to run IBM software on AWS services as a SaaS
• IBM consulting
  • close relationship
• Red Hat
  • many customers run redhat openshift on AWS (red hat is by )
• IBM technology
  • working together to generate products

ai governance

• AI Policies in Singapore regulation
  • Singapore Model Governance Framework (2019, 2020)
  • AI Verify - AI Governance Testing Framework + Toolkit (2022)
    • Technical model testing toolkit for explainability, fairness
    • https://file.go.gov.sg/aiverify.pdf
    • https://file.go.gov.sg/aiverify-primer.pdf
• AI biases leading to regulations
• SR11-7
  • risk model assets
  • financial services AI in the EU

IBM Cloud Pak for Data

• cpd-zen-xx
• has a notebook interface similar to sagemaker
• what is the diff between sagemaker and cloudpak?
  • ai governance ⇒ they added things for explainability etc
  • explainability ⇒ using SHAP and LIME

in RI-SageMaker-Deploy-Wstudio.ipynb:

• using sagemaker.LinearLearner to train on sagemaker
• does IBM have their own model, and way to conduct training too? why use sagemaker via cloudpak?
• can deploy it on other places also, currently uses sagemaker to deploy on the endpoint, then using ibm openscale to do drift detection

GitHub - IBM/watson-openscale-samples: Watson Openscale sample assets, notebooks and apps.

Trusted AI

• explainability ⇒ LIME, SHAP
• drift monitoring ⇒ IBM Documentation
• fairness ⇒ disparate impact
• quality ⇒ standard metrics - Quality metrics overview - IBM Documentation

Governance

• seems more on tools than any specific method / theory to do data governance

they mainly support tabular form

• if unstructured data - they need to do a transform first into the tabular form

charging / fees / liscencing

• based on worker nodes

They have some work with federated learning too IBM Documentation

---