Skip to Content
0 Votes

Changes for page Friday 14 February 2025

Last modified by dennis yoshikawa on 2025/02/14 01:23
From version 6.5
edited by dennis yoshikawa
on 2025/02/14 01:18
Change comment: There is no comment for this version
To version 4.18
edited by Haekal Yusril Faizin
on 2025/02/13 12:25
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.dennis
1 +XWiki.haekalfaizin
Content
... ... @@ -10,47 +10,25 @@
10 10  What Have You Done?
11 11  {{/success}}
12 12  
13 -* Airflow-airbyte maintenance
13 +1. xxx
14 +1. xxx
15 +1. xxx
14 14  
15 -~1. create task for delete gcs folder table dharmadexa
16 -2. ⁠change incremental method in airbyte for tb)
17 -
18 -* Dkonsul
19 -
20 -~1. Deploy transformation code to production.
21 -2. Final QC with partnership team.
22 -
23 -* ⁠*D2D*
24 -
25 -~1. enhance logic stickness rate (mart)
26 -2. ⁠weekly regrup w/product
27 -
28 -* MCN
29 -
30 -~1. troubleshoot on DAG and resource
31 -2. ⁠repointing new database for all pipeline existing
32 -
33 -* Screening
34 -
35 -~1. regroup w/PM and DA for validate data and discuss funneling data
36 -2. enhance logic fact table dharmadexa
37 -3. ⁠preparation dharmadexa data phase 3 (funneling)
38 -4. ⁠testing geo.py to get city from long lat
39 -5. ⁠migrasi dataset cleaned_screening_merck.dim_screening_stunting to CH
40 -
41 41  {{warning}}
42 42  What Issues You Have?
43 43  {{/warning}}
44 44  
45 -1. That's difficulty in finding open source data quality tools is that most of them are paid.
21 +1. xxx
22 +1. xxx
23 +1. xxx
46 46  
47 47  {{info}}
48 48  What Next You Will Do? (Optional)
49 49  {{/info}}
50 50  
51 -1. Exp
52 52  1. xxx
53 53  1. xxx
31 +1. xxx
54 54  
55 55  **What Support You Need? (Optional)**
56 56  
... ... @@ -113,18 +113,18 @@
113 113  
114 114  **AUTOMARK**
115 115  
116 -1. Deploy, evaluate and making documentation of **Screening Dharma Dexa API** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users_dd>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API could be accessed using prompt** to retrieve user that relevant to the prompt. Per 13 Feb 2024, **the RAG Accuracy is 87,50%**
94 +1. Deploy, evaluate and making documentation of **Screening Dharma Dexa API** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users_dd>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. The API could be accessed using prompt to retrieve user that relevant to tPer 13 Feb 2024, **the RAG Accuracy is 87,50%**
117 117  1. Deploy, evaluate and making documentation of **master user GUE Ecosystem** to be accessed by Tech team. The API is deployed at [[https:~~/~~/datalake.ptgue.com/v1/users>>https://datalake.ptgue.com/v1/users_dd]] and the **documentation is already sent to Tech team**. **The API is already tested by Tech team and have no issues.**
118 118  
119 119  **MCN**
120 120  
121 -1. Fixing resource spike issue caused by scraping schedule. The issue is already fixed and pass the test for 3 days (wednesday, thursday, and friday). **Scraping time is decreased from 1 minute per user to the maximum of +-20 seconds per user.** **(Pairing with Syifa-DE)**
122 -1. **Repoint, re-align, and redesign the pipeline and database** that being consumed for scrapping.** (Pairing with Syifa-DE)**
99 +1. Fixing resource spike issue caused by scraping schedule. The issue is already fixed and pass the test for 3 days (wednesday, thursday, and friday). Scraping time is decreased from 1 minute per user to the maximum of +-20 seconds per user. (Pairing with Syifa-DE)
100 +1. Repoint, re-align, and redesign the pipeline and database that being consumed for scrapping. (Pairing with Syifa-DE)
123 123  
124 124  **Automation**
125 125  
126 -1. **Implement compliance mapping** with the newest data from compliance team
127 -1. Data, flow, and script validation for user report performance doctor. **The sample report is already validated by dr.Astrid**
104 +1. Implement compliance mapping with the newest data from compliance team
105 +1. Data, flow, and script validation for user report performance doctor. The report sample is already validated by dr.Astrid.
128 128  
129 129  **Others**
130 130  
... ... @@ -136,53 +136,19 @@
136 136  What Issues You Have?
137 137  {{/warning}}
138 138  
139 -1. There is no info regarding database repointing of MCN from tech team. So, the data is not updated. Already solved by coordinating with Product Team
140 -1. Need to enhance the RAG accuracy to around 90%
117 +1. xxx
118 +1. xxx
119 +1. xxx
141 141  
142 142  {{info}}
143 143  What Next You Will Do? (Optional)
144 144  {{/info}}
145 145  
146 -1. Increase RAG accuracy by adding more train query LLM
147 -1. Start daily recurring scraping for tiktok dashboard
148 -1. Start dkonsul insight next week
125 +1. xxx
126 +1. xxx
127 +1. xxx
149 149  
150 150  **What Support You Need? (Optional)**
151 151  
152 -
153 -=== Summary ===
154 -
155 -Berikut ringkasan laporan mingguan dari 14 Februari 2025 yang ditulis oleh Haekal Yusril Faizin. Laporan ini merangkum aktivitas dan isu dari tim Data Engineer, Data Analyst, dan Data Analyst & AI.
156 -
157 -**Data Analyst**
158 -
159 -* Menyelesaikan fase kedua Dharma Dexa.
160 -* Memindahkan visualisasi data dan analitik dari Looker ke Metabase.
161 -* Mendesain ulang data DKonsul untuk struktur dan organisasi yang lebih baik.
162 -* Menangani berbagai permintaan data on-demand.
163 -
164 -**Data Analyst & AI**
165 -
166 -* Menerapkan, mengevaluasi, dan mendokumentasikan API Screening Dharma Dexa untuk diakses oleh tim Tech.
167 -* Menerapkan, mengevaluasi, dan mendokumentasikan master user GUE Ecosystem untuk diakses oleh tim Tech.
168 -* Memperbaiki masalah lonjakan sumber daya yang disebabkan oleh jadwal scraping.
169 -* Merepoint, menyelaraskan ulang, dan mendesain ulang pipeline dan database yang digunakan untuk scraping.
170 -* Menerapkan pemetaan kepatuhan dengan data terbaru dari tim kepatuhan.
171 -* Memvalidasi data, alur, dan skrip untuk kinerja laporan pengguna dokter.
172 -
173 -**Isu**
174 -
175 -* Pilihan bagan dan fleksibilitas yang terbatas dalam kustomisasi Metabase.
176 -* Kurangnya informasi mengenai database repointing MCN dari tim teknologi.
177 -* Perlu meningkatkan akurasi RAG menjadi sekitar 90%.
178 -
179 -**Langkah Selanjutnya**
180 -
181 -* Melanjutkan desain ulang data DKonsul dan migrasi ke Metabase.
182 -* Melanjutkan dengan Dharma Dexa Fase 3 dan mengembangkan Dasbor Pelacak Kunjungan AppSheet MCN.
183 -* Meningkatkan akurasi RAG, memulai scraping berulang harian untuk dasbor TikTok, dan memulai wawasan DKonsul minggu depan.
184 -
185 -**Dukungan yang Dibutuhkan**
186 -
187 -* Validasi data dan peningkatan penyaringan Dharma Dexa.
131 +
188 188  )))