Algorithm for monitoring social network data for behavioral economics research
Social network data analysis is an important tool for behavioral economics research. With the help of application programming interfaces (APIs) of various platforms, such as Facebook, Instagram, TikTok and others, a representative database is collected that allows content analysis. The proposed monitoring algorithm allows for a systematic analysis of the interaction of social network users.
According to the 2023 Global Digital report, more than 4.76 billion people use social networks. The most popular social networks by the number of active users (in millions) are: Facebook - 2958; YouTube - 2,514; WhatsApp - 2000; Instagram - 2000; WeChat – 1309 [1]. This is a powerful representative base for behavioral economics research. User responses are unbiased as they express their own thoughts and feelings, which is important for monitoring. Possession of modern methods, innovative and information technologies is becoming a requirement of the time. One of the most effective social media monitoring methods for behavioral economics research is the application programming interface (API). Public, private, or native identifiers for networks such as Facebook, Instagram, X, TikTok, and LinkedIn may receive a variety of data such as public user profiles, posts, comments, likes, user interactions, and other data that may be useful for research in behavioral economics and other fields.
Note that monitoring tasks for automatic data updates are generally available for all networks. This includes the ability to create tasks for one-time or automatic data updates. An «Update» is a simple task in the item’s one-time update queue. «Auto-Update» is a task to monitor items over a period of time. When creating an auto-update task, we recommend specifying parameters such as auto_update_interval — the data update interval and the auto_update_expire_at parameter, which specifies when automatic data monitoring should stop. It is also worth noting that this feature can be customized for all items such as posts, profiles, groups, post search, profile search, job search, etc. [2].
The API method allows you to search by keywords, analyze posts, content, comments, articles, interests, likes, distribution, number of reactions by type, by language, by date. Posts can be filtered by sort type (newest, most relevant), publication date (last month, last week, last 24 hours), language, and more.
We have developed an algorithm for monitoring data from social networks for research in behavioral economics, which includes ten steps:
- Collection of information from social networks by thematic requests.
- Raw data parsing using the API.
- Access data using an encrypted token.
- Development of individual algorithms for data collection from each social network.
- Creation of terminological bases for generating information search requests.
- Development of individual scripts for creating requests using the HTTP POST method in Python.
- Development of individual scripts for checking the status of requests using the http GET method in Python.
- Development of individual scripts for data preprocessing, transfer from cloud database pgadmin to local files in csv format.
- Coordination with the system of cyclic execution of operations taking into account API restrictions.
- Development of individual scripts for data analysis for each social network.
Thus, the use of API is a unique method for conducting research in behavioral economics. This method allows you to analyze the reactions of users in real time and study their thoughts, impressions and views. The monitoring algorithm developed by us allows you to systematically collect and analyze data from various social network platforms, which contributes to increasing the reliability of research results in behavioral economics.
References
- Digital 2023: Global Overview Report. (2023). Retrieved from https://datareportal.com/reports/digital-2023-global-overview-report
- Data 365 Social Media API Documentation (2023). Retrieved from https://data365.co/guides