Skip to Content
0 Votes

Changes for page Friday 14 February 2025

Last modified by dennis yoshikawa on 2025/02/14 01:23
From version 2.3
edited by Abdurachman Putra
on 2025/02/13 09:42
Change comment: There is no comment for this version
To version 8.1
edited by dennis yoshikawa
on 2025/02/14 01:23
Change comment: There is no comment for this version

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -Main.PT GUE.WebHome
1 +Weekly Report.February .WebHome
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.surya
1 +XWiki.dennis
Content
... ... @@ -1,58 +1,107 @@
1 1  = Weekly Report 14 February 2025 =
2 2  
3 3  
4 -== Data Engineer ==
4 +== (% id="cke_bm_29973S" style="display:none" %) (%%)Data Engineer ==
5 5  
6 6  >What Have You Done in This Week?
7 7  
8 8  (((
9 +{{success}}
9 9  What Have You Done?
11 +{{/success}}
10 10  
11 -1. xxx
12 -1. xxx
13 -1. xxx
13 +* Airflow-airbyte maintenance
14 14  
15 +~1. create task for delete gcs folder table dharmadexa
16 +2. ⁠change incremental method in airbyte for tb)
17 +
18 +* Dkonsul
19 +
20 +~1. Doctor incentive - Deploy transformation code to production.
21 +2. Doctor incentive - Final QC with partnership team. 
22 +3. Dataset migrate dataset for dashboard (Herbawell Dashboard, BRI Medika DKONSUL)
23 +
24 +* ⁠*D2D*
25 +
26 +~1. enhance logic stickness rate (mart)
27 +2. ⁠weekly regrup w/product
28 +
29 +* MCN
30 +
31 +~1. troubleshoot on DAG and resource
32 +2. ⁠repointing new database for all pipeline existing
33 +
34 +* Screening
35 +
36 +~1. regroup w/PM and DA for validate data and discuss funneling data
37 +2. enhance logic fact table dharmadexa
38 +3. ⁠preparation dharmadexa data phase 3 (funneling)
39 +4. ⁠testing geo.py to get city from long lat
40 +5. ⁠migrasi dataset cleaned_screening_merck.dim_screening_stunting to CH
41 +
42 +{{warning}}
15 15  What Issues You Have?
44 +{{/warning}}
16 16  
17 -1. xxx
18 -1. xxx
19 -1. xxx
46 +1. That's difficulty in finding open source data quality tools is that most of them are paid.
20 20  
48 +{{info}}
21 21  What Next You Will Do? (Optional)
50 +{{/info}}
22 22  
23 -1. xxx
24 -1. xxx
25 -1. xxx
52 +1. Explore more for finding open source data quality tools
53 +1. Tuning query for reduce cost bigquery
26 26  
27 -What Support You Need? (Optional)
55 +**What Support You Need? (Optional)**
56 +
57 +
58 +----
59 +
60 +
28 28  )))
29 29  
30 -== Data Analyst ==
31 31  
64 +== (% id="cke_bm_29973S" style="display:none" %) (%%)Data Analyst ==
65 +
32 32  >What Have You Done in This Week?
33 33  
34 34  (((
69 +{{success}}
35 35  What Have You Done?
71 +{{/success}}
36 36  
37 -1. xxx
38 -1. xxx
39 -1. xxx
73 +1. **Dharma Dexa Phase 2** – Completed the second phase of Dharma Dexa.
74 +1. **Migration from Looker to Metabase** – Transitioning data visualization and analytics from Looker to Metabase.
75 +1. **Redesigned DKonsul Data** – Improved the structure and organization of DKonsul data.
76 +1. **Ad-hoc Requests** – Handled various on-demand data requests.
40 40  
78 +{{warning}}
41 41  What Issues You Have?
80 +{{/warning}}
42 42  
43 -1. xxx
44 -1. xxx
45 -1. xxx
82 +1. **Metabase Limitations** – Limited chart options and flexibility in customization, particularly a lack of aggregation functions.
46 46  
84 +{{info}}
47 47  What Next You Will Do? (Optional)
86 +{{/info}}
48 48  
49 -1. xxx
50 -1. xxx
51 -1. xxx
88 +1. **Continue Redesigning DKonsul Data**
89 +1*. Refining the data funnel from consultation → prescription → transaction.
90 +1. **Continue Migration to Metabase** – Ensuring a smooth transition from Looker to Metabase.
91 +1. **Dharma Dexa Phase 3** – Proceeding with the next phase of the Dharma Dexa project.
92 +1. **AppSheet MCN Visit Tracker Dashboard** – Developing and optimizing the dashboard.
52 52  
53 -What Support You Need? (Optional)
94 +**What Support You Need? (Optional)**
95 +
96 +1. **Data Validation** – Ensuring data accuracy and consistency.
97 +1. **Dharma Dexa Screening Enhancements**
98 +1*. Assigning a **new screening ID** for each event, especially if different questions and inputs are involved.
99 +1*. Adding **location input (province, city)** for better analysis.
54 54  )))
55 55  
102 +
103 +----
104 +
56 56  == Data Analyst & AI ==
57 57  
58 58  >What Have You Done in This Week?
... ... @@ -62,27 +62,78 @@
62 62  What Have You Done?
63 63  {{/success}}
64 64  
65 -1. xxx
66 -1. xxx
67 -1. xxx
114 +**AUTOMARK**
68 68  
116 +1. Deploy, evaluate and making documentation of **Screening Dharma Dexa API** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users_dd>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API could be accessed using prompt** to retrieve user that relevant to the prompt. Per 13 Feb 2024, **the RAG Accuracy is 87,50%**
117 +1. Deploy, evaluate and making documentation of **master user GUE Ecosystem** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API is already tested by Tech team and have no issues.**
118 +
119 +**MCN**
120 +
121 +1. Fixing resource spike issue caused by scraping schedule. The issue is already fixed and pass the test for 3 days (wednesday, thursday, and friday). **Scraping time is decreased from 1 minute per user to the maximum of +-20 seconds per user.** **(Pairing with Syifa-DE)**
122 +1. **Repoint, re-align, and redesign the pipeline and database** that being consumed for scrapping.** (Pairing with Syifa-DE)**
123 +
124 +**Automation**
125 +
126 +1. **Implement compliance mapping** with the newest data from compliance team
127 +1. Data, flow, and script validation for user report performance doctor. **The sample report is already validated by dr.Astrid**
128 +
129 +**Others**
130 +
131 +1. Reqeust + enhance dashboard merck
132 +1. Request dkonsul data for online doctors, transactions, prescriptions, Dexa prescriptions, comparison new users january
133 +1. Request data ICD-10
134 +
69 69  {{warning}}
70 70  What Issues You Have?
71 71  {{/warning}}
72 72  
73 -1. xxx
74 -1. xxx
75 -1. xxx
139 +1. There is no info regarding database repointing of MCN from tech team. So, the data is not updated. Already solved by coordinating with Product Team
140 +1. Need to enhance the RAG accuracy to around 90%
76 76  
77 77  {{info}}
78 78  What Next You Will Do? (Optional)
79 79  {{/info}}
80 80  
81 -1. xxx
82 -1. xxx
83 -1. xxx
146 +1. Increase RAG accuracy by adding more train query LLM
147 +1. Start daily recurring scraping for tiktok dashboard
148 +1. Start dkonsul insight next week
84 84  
85 -What Support You Need? (Optional)
150 +**What Support You Need? (Optional)**
86 86  
87 -
152 +
153 +=== Summary ===
154 +
155 +Berikut ringkasan laporan mingguan dari 14 Februari 2025 yang ditulis oleh Haekal Yusril Faizin. Laporan ini merangkum aktivitas dan isu dari tim Data Engineer, Data Analyst, dan Data Analyst & AI.
156 +
157 +**Data Analyst**
158 +
159 +* Menyelesaikan fase kedua Dharma Dexa.
160 +* Memindahkan visualisasi data dan analitik dari Looker ke Metabase.
161 +* Mendesain ulang data DKonsul untuk struktur dan organisasi yang lebih baik.
162 +* Menangani berbagai permintaan data on-demand.
163 +
164 +**Data Analyst & AI**
165 +
166 +* Menerapkan, mengevaluasi, dan mendokumentasikan API Screening Dharma Dexa untuk diakses oleh tim Tech.
167 +* Menerapkan, mengevaluasi, dan mendokumentasikan master user GUE Ecosystem untuk diakses oleh tim Tech.
168 +* Memperbaiki masalah lonjakan sumber daya yang disebabkan oleh jadwal scraping.
169 +* Merepoint, menyelaraskan ulang, dan mendesain ulang pipeline dan database yang digunakan untuk scraping.
170 +* Menerapkan pemetaan kepatuhan dengan data terbaru dari tim kepatuhan.
171 +* Memvalidasi data, alur, dan skrip untuk kinerja laporan pengguna dokter.
172 +
173 +**Isu**
174 +
175 +* Pilihan bagan dan fleksibilitas yang terbatas dalam kustomisasi Metabase.
176 +* Kurangnya informasi mengenai database repointing MCN dari tim teknologi.
177 +* Perlu meningkatkan akurasi RAG menjadi sekitar 90%.
178 +
179 +**Langkah Selanjutnya**
180 +
181 +* Melanjutkan desain ulang data DKonsul dan migrasi ke Metabase.
182 +* Melanjutkan dengan Dharma Dexa Fase 3 dan mengembangkan Dasbor Pelacak Kunjungan AppSheet MCN.
183 +* Meningkatkan akurasi RAG, memulai scraping berulang harian untuk dasbor TikTok, dan memulai wawasan DKonsul minggu depan.
184 +
185 +**Dukungan yang Dibutuhkan**
186 +
187 +* Validasi data dan peningkatan penyaringan Dharma Dexa.
88 88  )))