Changes for page Friday 14 February 2025
Last modified by dennis yoshikawa on 2025/02/14 01:23
From version 
2.4 
2.4
edited by Abdurachman Putra
on 2025/02/13 09:43
on 2025/02/13 09:43
Change comment:
There is no comment for this version
To version 8.1
8.1
edited by dennis yoshikawa
on 2025/02/14 01:23
on 2025/02/14 01:23
Change comment:
There is no comment for this version
Show context
Hide context
Summary
Details
-
Page properties
-
- Parent
-
... ... @@ -1,1 +1,1 @@ 1 - Main.PTGUE.WebHome1 +Weekly Report.February .WebHome - Author
-
... ... @@ -1,1 +1,1 @@ 1 -XWiki.s urya1 +XWiki.dennis - Content
-
... ... @@ -10,112 +10,179 @@ 10 10 What Have You Done? 11 11 {{/success}} 12 12 13 -1. xxx 14 -1. xxx 15 -1. xxx 13 +* Airflow-airbyte maintenance 16 16 15 +~1. create task for delete gcs folder table dharmadexa 16 +2. change incremental method in airbyte for tb) 17 + 18 +* Dkonsul 19 + 20 +~1. Doctor incentive - Deploy transformation code to production. 21 +2. Doctor incentive - Final QC with partnership team. 22 +3. Dataset migrate dataset for dashboard (Herbawell Dashboard, BRI Medika DKONSUL) 23 + 24 +* *D2D* 25 + 26 +~1. enhance logic stickness rate (mart) 27 +2. weekly regrup w/product 28 + 29 +* MCN 30 + 31 +~1. troubleshoot on DAG and resource 32 +2. repointing new database for all pipeline existing 33 + 34 +* Screening 35 + 36 +~1. regroup w/PM and DA for validate data and discuss funneling data 37 +2. enhance logic fact table dharmadexa 38 +3. preparation dharmadexa data phase 3 (funneling) 39 +4. testing geo.py to get city from long lat 40 +5. migrasi dataset cleaned_screening_merck.dim_screening_stunting to CH 41 + 17 17 {{warning}} 18 18 What Issues You Have? 19 19 {{/warning}} 20 20 21 -1. xxx 22 -1. xxx 23 -1. xxx 46 +1. That's difficulty in finding open source data quality tools is that most of them are paid. 24 24 25 25 {{info}} 26 26 What Next You Will Do? (Optional) 27 27 {{/info}} 28 28 29 -1. xxx 30 -1. xxx 31 -1. xxx 52 +1. Explore more for finding open source data quality tools 53 +1. Tuning query for reduce cost bigquery 32 32 33 33 **What Support You Need? (Optional)** 56 + 57 + 58 +---- 59 + 60 + 34 34 ))) 35 35 36 36 37 -== Data Engineer==64 +== (% id="cke_bm_29973S" style="display:none" %) (%%)Data Analyst == 38 38 39 39 >What Have You Done in This Week? 40 40 41 41 ((( 69 +{{success}} 42 42 What Have You Done? 71 +{{/success}} 43 43 44 -1. xxx 45 -1. xxx 46 -1. xxx 73 +1. **Dharma Dexa Phase 2** – Completed the second phase of Dharma Dexa. 74 +1. **Migration from Looker to Metabase** – Transitioning data visualization and analytics from Looker to Metabase. 75 +1. **Redesigned DKonsul Data** – Improved the structure and organization of DKonsul data. 76 +1. **Ad-hoc Requests** – Handled various on-demand data requests. 47 47 78 +{{warning}} 48 48 What Issues You Have? 80 +{{/warning}} 49 49 50 -1. xxx 51 -1. xxx 52 -1. xxx 82 +1. **Metabase Limitations** – Limited chart options and flexibility in customization, particularly a lack of aggregation functions. 53 53 84 +{{info}} 54 54 What Next You Will Do? (Optional) 86 +{{/info}} 55 55 56 -1. xxx 57 -1. xxx 58 -1. xxx 88 +1. **Continue Redesigning DKonsul Data** 89 +1*. Refining the data funnel from consultation → prescription → transaction. 90 +1. **Continue Migration to Metabase** – Ensuring a smooth transition from Looker to Metabase. 91 +1. **Dharma Dexa Phase 3** – Proceeding with the next phase of the Dharma Dexa project. 92 +1. **AppSheet MCN Visit Tracker Dashboard** – Developing and optimizing the dashboard. 59 59 60 -What Support You Need? (Optional) 94 +**What Support You Need? (Optional)** 95 + 96 +1. **Data Validation** – Ensuring data accuracy and consistency. 97 +1. **Dharma Dexa Screening Enhancements** 98 +1*. Assigning a **new screening ID** for each event, especially if different questions and inputs are involved. 99 +1*. Adding **location input (province, city)** for better analysis. 61 61 ))) 62 62 63 -== Data Analyst == 64 64 103 +---- 104 + 105 +== Data Analyst & AI == 106 + 65 65 >What Have You Done in This Week? 66 66 67 67 ((( 110 +{{success}} 68 68 What Have You Done? 112 +{{/success}} 69 69 70 -1. xxx 71 -1. xxx 72 -1. xxx 114 +**AUTOMARK** 73 73 74 -What Issues You Have? 116 +1. Deploy, evaluate and making documentation of **Screening Dharma Dexa API** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users_dd>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API could be accessed using prompt** to retrieve user that relevant to the prompt. Per 13 Feb 2024, **the RAG Accuracy is 87,50%** 117 +1. Deploy, evaluate and making documentation of **master user GUE Ecosystem** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API is already tested by Tech team and have no issues.** 75 75 76 -1. xxx 77 -1. xxx 78 -1. xxx 119 +**MCN** 79 79 80 -What Next You Will Do? (Optional) 121 +1. Fixing resource spike issue caused by scraping schedule. The issue is already fixed and pass the test for 3 days (wednesday, thursday, and friday). **Scraping time is decreased from 1 minute per user to the maximum of +-20 seconds per user.** **(Pairing with Syifa-DE)** 122 +1. **Repoint, re-align, and redesign the pipeline and database** that being consumed for scrapping.** (Pairing with Syifa-DE)** 81 81 82 -1. xxx 83 -1. xxx 84 -1. xxx 124 +**Automation** 85 85 86 - WhatSupportYouNeed?(Optional)87 - )))126 +1. **Implement compliance mapping** with the newest data from compliance team 127 +1. Data, flow, and script validation for user report performance doctor. **The sample report is already validated by dr.Astrid** 88 88 89 - == Data Analyst & AI ==129 +**Others** 90 90 91 ->What Have You Done in This Week? 131 +1. Reqeust + enhance dashboard merck 132 +1. Request dkonsul data for online doctors, transactions, prescriptions, Dexa prescriptions, comparison new users january 133 +1. Request data ICD-10 92 92 93 -((( 94 -{{success}} 95 -What Have You Done? 96 -{{/success}} 97 - 98 -1. xxx 99 -1. xxx 100 -1. xxx 101 - 102 102 {{warning}} 103 103 What Issues You Have? 104 104 {{/warning}} 105 105 106 -1. xxx 107 -1. xxx 108 -1. xxx 139 +1. There is no info regarding database repointing of MCN from tech team. So, the data is not updated. Already solved by coordinating with Product Team 140 +1. Need to enhance the RAG accuracy to around 90% 109 109 110 110 {{info}} 111 111 What Next You Will Do? (Optional) 112 112 {{/info}} 113 113 114 -1. xxx115 -1. xxx116 -1. x xx146 +1. Increase RAG accuracy by adding more train query LLM 147 +1. Start daily recurring scraping for tiktok dashboard 148 +1. Start dkonsul insight next week 117 117 118 118 **What Support You Need? (Optional)** 119 119 120 - 152 + 153 +=== Summary === 154 + 155 +Berikut ringkasan laporan mingguan dari 14 Februari 2025 yang ditulis oleh Haekal Yusril Faizin. Laporan ini merangkum aktivitas dan isu dari tim Data Engineer, Data Analyst, dan Data Analyst & AI. 156 + 157 +**Data Analyst** 158 + 159 +* Menyelesaikan fase kedua Dharma Dexa. 160 +* Memindahkan visualisasi data dan analitik dari Looker ke Metabase. 161 +* Mendesain ulang data DKonsul untuk struktur dan organisasi yang lebih baik. 162 +* Menangani berbagai permintaan data on-demand. 163 + 164 +**Data Analyst & AI** 165 + 166 +* Menerapkan, mengevaluasi, dan mendokumentasikan API Screening Dharma Dexa untuk diakses oleh tim Tech. 167 +* Menerapkan, mengevaluasi, dan mendokumentasikan master user GUE Ecosystem untuk diakses oleh tim Tech. 168 +* Memperbaiki masalah lonjakan sumber daya yang disebabkan oleh jadwal scraping. 169 +* Merepoint, menyelaraskan ulang, dan mendesain ulang pipeline dan database yang digunakan untuk scraping. 170 +* Menerapkan pemetaan kepatuhan dengan data terbaru dari tim kepatuhan. 171 +* Memvalidasi data, alur, dan skrip untuk kinerja laporan pengguna dokter. 172 + 173 +**Isu** 174 + 175 +* Pilihan bagan dan fleksibilitas yang terbatas dalam kustomisasi Metabase. 176 +* Kurangnya informasi mengenai database repointing MCN dari tim teknologi. 177 +* Perlu meningkatkan akurasi RAG menjadi sekitar 90%. 178 + 179 +**Langkah Selanjutnya** 180 + 181 +* Melanjutkan desain ulang data DKonsul dan migrasi ke Metabase. 182 +* Melanjutkan dengan Dharma Dexa Fase 3 dan mengembangkan Dasbor Pelacak Kunjungan AppSheet MCN. 183 +* Meningkatkan akurasi RAG, memulai scraping berulang harian untuk dasbor TikTok, dan memulai wawasan DKonsul minggu depan. 184 + 185 +**Dukungan yang Dibutuhkan** 186 + 187 +* Validasi data dan peningkatan penyaringan Dharma Dexa. 121 121 )))
