Skip to Content
0 Votes

Changes for page Friday 14 February 2025

Last modified by dennis yoshikawa on 2025/02/14 01:23
From version 1.2
edited by Abdurachman Putra
on 2025/02/13 09:36
Change comment: There is no comment for this version
To version 6.3
edited by dennis yoshikawa
on 2025/02/14 01:14
Change comment: There is no comment for this version

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -Main.PT GUE.WebHome
1 +Weekly Report.February .WebHome
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.surya
1 +XWiki.dennis
Content
... ... @@ -1,6 +1,190 @@
1 1  = Weekly Report 14 February 2025 =
2 2  
3 3  
4 -== Data Engineer ==
4 +== (% id="cke_bm_29973S" style="display:none" %) (%%)Data Engineer ==
5 5  
6 ->What Have You Done?/What Issues You Have?What Next You Will Do? (Optional)What Support You Need? (Optional)
6 +>What Have You Done in This Week?
7 +
8 +(((
9 +{{success}}
10 +What Have You Done?
11 +{{/success}}
12 +
13 +* Airflow-airbyte maintenance
14 +
15 +~1. create task for delete gcs folder table dharmadexa
16 +2. ⁠change incremental method in airbyte for tb)
17 +
18 +* Dkonsul
19 +
20 +~1. Deploy transformation code to production
21 +2. 
22 +
23 +* ⁠*D2D*
24 +
25 +~1. enhance logic stickness rate (mart)
26 +2. ⁠weekly regrup w/product
27 +
28 +* MCN
29 +
30 +~1. troubleshoot on DAG and resource
31 +2. ⁠repointing new database for all pipeline existing
32 +
33 +* Screening
34 +
35 +~1. regroup w/PM and DA for validate data and discuss funneling data
36 +2. enhance logic fact table dharmadexa
37 +3. ⁠preparation dharmadexa data phase 3 (funneling)
38 +4. ⁠testing geo.py to get city from long lat
39 +5. ⁠migrasi dataset cleaned_screening_merck.dim_screening_stunting to CH
40 +
41 +{{warning}}
42 +What Issues You Have?
43 +{{/warning}}
44 +
45 +1. xxx
46 +1. xxx
47 +1. xxx
48 +
49 +{{info}}
50 +What Next You Will Do? (Optional)
51 +{{/info}}
52 +
53 +1. xxx
54 +1. xxx
55 +1. xxx
56 +
57 +**What Support You Need? (Optional)**
58 +
59 +
60 +----
61 +
62 +
63 +)))
64 +
65 +
66 +== (% id="cke_bm_29973S" style="display:none" %) (%%)Data Analyst ==
67 +
68 +>What Have You Done in This Week?
69 +
70 +(((
71 +{{success}}
72 +What Have You Done?
73 +{{/success}}
74 +
75 +1. **Dharma Dexa Phase 2** – Completed the second phase of Dharma Dexa.
76 +1. **Migration from Looker to Metabase** – Transitioning data visualization and analytics from Looker to Metabase.
77 +1. **Redesigned DKonsul Data** – Improved the structure and organization of DKonsul data.
78 +1. **Ad-hoc Requests** – Handled various on-demand data requests.
79 +
80 +{{warning}}
81 +What Issues You Have?
82 +{{/warning}}
83 +
84 +1. **Metabase Limitations** – Limited chart options and flexibility in customization, particularly a lack of aggregation functions.
85 +
86 +{{info}}
87 +What Next You Will Do? (Optional)
88 +{{/info}}
89 +
90 +1. **Continue Redesigning DKonsul Data**
91 +1*. Refining the data funnel from consultation → prescription → transaction.
92 +1. **Continue Migration to Metabase** – Ensuring a smooth transition from Looker to Metabase.
93 +1. **Dharma Dexa Phase 3** – Proceeding with the next phase of the Dharma Dexa project.
94 +1. **AppSheet MCN Visit Tracker Dashboard** – Developing and optimizing the dashboard.
95 +
96 +**What Support You Need? (Optional)**
97 +
98 +1. **Data Validation** – Ensuring data accuracy and consistency.
99 +1. **Dharma Dexa Screening Enhancements**
100 +1*. Assigning a **new screening ID** for each event, especially if different questions and inputs are involved.
101 +1*. Adding **location input (province, city)** for better analysis.
102 +)))
103 +
104 +
105 +----
106 +
107 +== Data Analyst & AI ==
108 +
109 +>What Have You Done in This Week?
110 +
111 +(((
112 +{{success}}
113 +What Have You Done?
114 +{{/success}}
115 +
116 +**AUTOMARK**
117 +
118 +1. Deploy, evaluate and making documentation of **Screening Dharma Dexa API** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users_dd>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API could be accessed using prompt** to retrieve user that relevant to the prompt. Per 13 Feb 2024, **the RAG Accuracy is 87,50%**
119 +1. Deploy, evaluate and making documentation of **master user GUE Ecosystem** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API is already tested by Tech team and have no issues.**
120 +
121 +**MCN**
122 +
123 +1. Fixing resource spike issue caused by scraping schedule. The issue is already fixed and pass the test for 3 days (wednesday, thursday, and friday). **Scraping time is decreased from 1 minute per user to the maximum of +-20 seconds per user.** **(Pairing with Syifa-DE)**
124 +1. **Repoint, re-align, and redesign the pipeline and database** that being consumed for scrapping.** (Pairing with Syifa-DE)**
125 +
126 +**Automation**
127 +
128 +1. **Implement compliance mapping** with the newest data from compliance team
129 +1. Data, flow, and script validation for user report performance doctor. **The sample report is already validated by dr.Astrid**
130 +
131 +**Others**
132 +
133 +1. Reqeust + enhance dashboard merck
134 +1. Request dkonsul data for online doctors, transactions, prescriptions, Dexa prescriptions, comparison new users january
135 +1. Request data ICD-10
136 +
137 +{{warning}}
138 +What Issues You Have?
139 +{{/warning}}
140 +
141 +1. There is no info regarding database repointing of MCN from tech team. So, the data is not updated. Already solved by coordinating with Product Team
142 +1. Need to enhance the RAG accuracy to around 90%
143 +
144 +{{info}}
145 +What Next You Will Do? (Optional)
146 +{{/info}}
147 +
148 +1. Increase RAG accuracy by adding more train query LLM
149 +1. Start daily recurring scraping for tiktok dashboard
150 +1. Start dkonsul insight next week
151 +
152 +**What Support You Need? (Optional)**
153 +
154 +
155 +=== Summary ===
156 +
157 +Berikut ringkasan laporan mingguan dari 14 Februari 2025 yang ditulis oleh Haekal Yusril Faizin. Laporan ini merangkum aktivitas dan isu dari tim Data Engineer, Data Analyst, dan Data Analyst & AI.
158 +
159 +**Data Analyst**
160 +
161 +* Menyelesaikan fase kedua Dharma Dexa.
162 +* Memindahkan visualisasi data dan analitik dari Looker ke Metabase.
163 +* Mendesain ulang data DKonsul untuk struktur dan organisasi yang lebih baik.
164 +* Menangani berbagai permintaan data on-demand.
165 +
166 +**Data Analyst & AI**
167 +
168 +* Menerapkan, mengevaluasi, dan mendokumentasikan API Screening Dharma Dexa untuk diakses oleh tim Tech.
169 +* Menerapkan, mengevaluasi, dan mendokumentasikan master user GUE Ecosystem untuk diakses oleh tim Tech.
170 +* Memperbaiki masalah lonjakan sumber daya yang disebabkan oleh jadwal scraping.
171 +* Merepoint, menyelaraskan ulang, dan mendesain ulang pipeline dan database yang digunakan untuk scraping.
172 +* Menerapkan pemetaan kepatuhan dengan data terbaru dari tim kepatuhan.
173 +* Memvalidasi data, alur, dan skrip untuk kinerja laporan pengguna dokter.
174 +
175 +**Isu**
176 +
177 +* Pilihan bagan dan fleksibilitas yang terbatas dalam kustomisasi Metabase.
178 +* Kurangnya informasi mengenai database repointing MCN dari tim teknologi.
179 +* Perlu meningkatkan akurasi RAG menjadi sekitar 90%.
180 +
181 +**Langkah Selanjutnya**
182 +
183 +* Melanjutkan desain ulang data DKonsul dan migrasi ke Metabase.
184 +* Melanjutkan dengan Dharma Dexa Fase 3 dan mengembangkan Dasbor Pelacak Kunjungan AppSheet MCN.
185 +* Meningkatkan akurasi RAG, memulai scraping berulang harian untuk dasbor TikTok, dan memulai wawasan DKonsul minggu depan.
186 +
187 +**Dukungan yang Dibutuhkan**
188 +
189 +* Validasi data dan peningkatan penyaringan Dharma Dexa.
190 +)))