MediaWiki talk:Architecture of SAR service

From Wikibase Personal data
Jump to navigation Jump to search

This is a good place for User:Haixinshi to discuss his progress Podehaye (talk)

7.27(Le Temps, Watson Actu)

Progress:

  1. MP suggested me to have a look at: VKontakte SDK and Tencent SDK in Watson Actu:
  • smali_classes2/com/vk/api/sdk/okhttp/OkHttpExecutor.smali and the function is ReadResponse. Unfortunately this function was not called during my experiments. I also tried other functions related to server response in VKontakte SDK. But they are not called.
  • I found that COMSCORE SDK was called very frequently in Watson Actu.
  • Bidding related functions from Google Ads mentioned on 7.26 are not called during my experiments.
  1. I finished saving detailed logs about LeTemps+AppNexus, which contains a lot of interesting information.

=== Questions: ===I

  1. Always finding functions by purely viewing logging is in very low efficiency. I will try to use more advanced tools, like Android Studio, so as to dynamically debug the modified APK if applicable, which may efficiently help me seize what kind of functions are actually called. Would I have any technical supports related to this?
  2. During working, I spent a lot of time in testing that whether interesting functions are called or not while focusing on some interesting strings like "bidding", and "price". It can lead to low productivity and boredom.

Plans:

  1. Find more efficient tools or platforms to do dynamical analysis.
  2. Write static analysis about SDK connections in Le Temps.
  3. Dynamically test COMSCORE SDK, VKontakte SDK and Tencent SDK in Watson Actu.

7.26(Le Temps, Watson Actu)

Progress:

  1. Have tried to dynamically modify and test all SDK listed on 7.23, but they are not called(Almost all ads are from AppNexus, which has been done in last week). Though I can do static analysis, it is not guaranteed for working in the future and can not be visualized.
  2. Understand the advertisement mechanism in Le Temps.
  3. After reviewing the apps listed by MP, I selected Watson Actu, since it contains much more diverse types of advertisements(including banner, Video Ads, etc see here), and such ads are frequently showing in this app.
  4. Decompile Watson Actu and find bidding functions from Google Ads, though they are heavily obfuscated:
  • smali_classes2/com/google/android/gms/internal/ads/zzezz.smali: const-string v10, "bid_response"
  • smali_classes2/com/google/android/gms/internal/ads/zzfac.smali: const-string v11, "bidding_data"
  • smali_classes2/com/google/android/gms/internal/ads/zzdxq.smali: const-string v1, "biddingData"
  • smali_classes2/com/google/android/gms/internal/ads/zzbjl.smali: const-string v3, "gads:scar_trustless_token_for_gbid:enabled"
  • smali_classes2/com/google/android/gms/internal/ads/zzbjl.smali: const-string v2, "gads:inspector:bidding_data_enabled"

Questions:

  1. Is the static design analysis report desirable? I am afraid that such reports can offer little help in practice usage. My preference is to dynamically analyze and store data. But the biggest obstacle for me is to find the functions or related advertisement companies that will be frequently called or activated during app running, which will help me dynamically analyze them. I need to get such suitable apps.
  2. I need more concrete requirements, which can be from the experience of trying mobile apps. For example, you try to use some apps, and then you can find some data showing on UI that interests you. By receiving you interests, I can do more, which is not limited to advertisement bidding.

Plan:

  1. I try to focus on Google ads by analyzing Watson Actu, which is wildly used.

7.25(Le Temps)

Progress:

  1. √ Make corresponding methods that can parse and save JSON and other information. This point is very important for me to improve efficiency. Previously, I have to write all smali inside a target function, which would not only cause register overlapping and control flow error, but also introduce complexity since one line Java code can be compiled into multiple lines small code. After reaching this point, in the future, I only need to write java code, which is more efficient, and then transform an integration into target smali code. It can speed up a lot!
  2. × Try to dynamically test SDKs in terms of advertisement.(I was stuck by point 4)
  3. × Try to find and collect message flow in Le Temps in terms of advertisement network. (I was stuck by point 4)
  4. √ Let Storing Data Locally work in different environments. Currently Storing Data Locally can work well in my Android Emulator, but it is more tricky to adapt to other environments like the Samsung Phone Paul gave me. The reasons are:
  • If there is no SdCard, we have to store data into internal memory. However, if the mobile app is not rooted, it is hard to access to data in internal memory(Users can not access to this data, and only the app itself can).
  • Now, I can get the correct absolute address to store data in internal storage and external storage, which can be adapted to any mobile phones.(But it is not convenient for me to debug on Samsung Phone, since I can not view text easily). So I will use Android Emulator in next steps for efficiency.

Questions:

  1. Can I root the phone? rooting is not reversible and it is very common in dev teams.
  2. The problem about Saving Data Locally is unavoidable in the future. I have a basic idea about how to design it. We can create a floating window for the users(users can hide it of course). In this window, users can view what type of advertisement they are exposed to; users can choose and send data to our server for further processing. But I am not very sure about this, because it seems like a heavy modification for an APP. The Game Testing Team in ByteDance would choose to use another APP to dynamically inject into targeting APP.

Plans:

  1. Try to dynamically test SDKs in terms of advertisement.
  2. Try to find and collect message flow in Le Temps in terms of advertisement network.

7.23(Le Temps)

Progress:

"Dynamic" means that when we run our modified APP and there is an advertisement showing, then the modified functions will be called and we can store dynamic data(especially advertisement price information).

SDK: AppNexus(Dynamically Tested)

SDK:AppNexus:UTAdResponse We can dynamically access to the following information: 

   private static final String RESPONSE_KEY_TAGS = "tags";
   private static final String RESPONSE_KEY_CONTENT = "content";
   private static final String RESPONSE_KEY_WIDTH = "width";
   private static final String RESPONSE_KEY_HEIGHT = "height";
   private static final String RESPONSE_KEY_PLAYER_WIDTH = "player_width";
   private static final String RESPONSE_KEY_PLAYER_HEIGHT = "player_height";
   private static final String RESPONSE_KEY_NO_BID = "nobid";
   private static final String RESPONSE_KEY_CREATIVE_ID = "creative_id";
   private static final String RESPONSE_KEY_ADS = "ads";
   private static final String RESPONSE_KEY_NOTIFY_URL = "notify_url";
   private static final String RESPONSE_KEY_CONTENT_SOURCE = "content_source";
   private static final String RESPONSE_KEY_CLASS = "class";
   private static final String RESPONSE_KEY_PARAM = "param";
   private static final String RESPONSE_KEY_PAYLOAD = "payload";
   private static final String RESPONSE_KEY_ID = "id";
   private static final String RESPONSE_KEY_UUID = "uuid";
   private static final String RESPONSE_KEY_HANDLER_URL = "url";
   private static final String RESPONSE_VALUE_ANDROID = "android";
   private static final String RESPONSE_KEY_TYPE = "type";
   private static final String RESPONSE_KEY_AD_TYPE = "ad_type";
   private static final String RESPONSE_KEY_HANDLER = "handler";
   private static final String RESPONSE_KEY_TRACKERS = "trackers";
   private static final String RESPONSE_KEY_IMPRESSION_URLS = "impression_urls";
   private static final String RESPONSE_KEY_CLICK_URLS = "click_urls";
   private static final String RESPONSE_KEY_ERROR_URLS = "error_urls";
   private static final String RESPONSE_KEY_TIMEOUT = "timeout_ms";
   private static final String RESPONSE_KEY_RESPONSE_URL = "response_url";
   private static final String RESPONSE_KEY_NO_AD_URL = "no_ad_url";
   private static final String RESPONSE_KEY_TAG_ID = "tag_id";
   private static final String RESPONSE_KEY_AUCTION_ID = "auction_id";
   private static final String RESPONSE_KEY_SECOND_PRICE = "second_price";
   private static final String RESPONSE_KEY_BUYER_MEMBER_ID = "buyer_member_id";
   private static final String RESPONSE_KEY_CPM = "cpm";
   private static final String RESPONSE_KEY_CPM_PUBLISHER_CURRENCY = "cpm_publisher_currency";
   private static final String RESPONSE_KEY_CPM_CURRENCY_CODE = "publisher_currency_code";

SDK: Magnite : Rubicon Advertising(No Dynamic Test)

It is not clear about what information we can collect from this API, but price for a banner ad can be obtained.

  1. RubiconHelper
  2. RubiconBanner

SDK: AmazonHB(No Dynamic Test)

We can collect price for an advertisement from Amazon:

  1. import com.amazon.device.ads.DTBAdResponse;

SDK: Smart AdServer(No Dynamic Test)

Bidding Example We can get bidding information from SASBiddingAdResponse.

  1. import com.smartadserver.android.library.headerbidding.SASBiddingAdResponse
  2. import com.smartadserver.android.library.headerbidding.SASBiddingFormatType
  3. import com.smartadserver.android.library.headerbidding.SASBiddingManager

SDK: Criteo(No Dynamic Test)

SDK Criteo : Bid involves price information. SDK Criteo : CdbResponseSlot involves a abundant information about bidding informtion: 

   @SerializedName("impId") val impressionId: String? = null,
   @SerializedName("placementId") val placementId: String? = null,
   @SerializedName("zoneId") val zoneId: Int? = null,
   @SerializedName("cpm") val cpm: String = "0.0",
   @SerializedName("currency") val currency: String? = null,
   @SerializedName("width") val width: Int = 0,
   @SerializedName("height") val height: Int = 0,
   @SerializedName("displayUrl") val displayUrl: String? = null,
   @SerializedName("native") val nativeAssets: NativeAssets? = null,
   @SerializedName("ttl") var ttlInSeconds: Int = 0,

SDK: Google ads(No Dynamic Test):

The following two can be matched according to data type:

  1. SDK:google.android.gem.ads : AdValue pulic reference code
  2. SDK:google.android.gem.ads : AdValue obfuscated code
  • String getCurrencyCode() //The value's ISO 4217 currency code.
  • int getPrecisionType() //The precision type of the reported ad value.
  • long getValueMicros()//The ad's value in micro-units, where 1,000,000 micro-units equal one unit of the currency.

Questions:

No.

Plans:

  1. Try to make a class and corresponding methods that can parse and save JSON and other information.
  2. Try to dynamically test SDKs in terms of advertisement.
  3. Try to find and collect message flow in Le Temps in terms of advertisement network.
Retrieved from "https://wiki.personaldata.io/w/index.php?title=MediaWiki_talk:Architecture_of_SAR_service&oldid=95994"