In this article, we review how to ingest, process, and analyze data from external sources using APIs and Python, using Binance's public API as an example to obtain prices for cryptocurrency pairs and filter them with the pandas and requests libraries.
The following is a complete Python script that implements this data ingestion. It is designed as a starting point for integrating it into data pipelines, automations, or for generating historical records from the Binance API.
import pandas as pd; import requests; BASE_URL = 'https://api.binance.com'; TARGET_PAIRS = ['BTCUSDT','ETHBTC','ETHUSDT','SOLUSDT']; def get_latest_prices(): # Gets, parses, and filters price data from the Binance API endpoint = f{BASE_URL}/api/v3/ticker/price; response = requests.get(endpoint); data = response.json(); price_df = pd.DataFrame(data); filtered_df = price_df[price_df['symbol'].isin(TARGET_PAIRS)]; print(filtered_df); return filtered_df; if __name__ == __main__: get_latest_prices()
Next, we break down the script step by step to understand its structure and points where it can be expanded for production.
1. Configuration and separation of values We define BASE_URL and TARGET_PAIRS at the beginning to separate configuration from logic. This makes it easier to modify target services or pairs without touching the main function and improves code maintainability.
2. Communication with the API using requests The requests library handles the HTTP transaction with the Binance API. With requests.get we point to the endpoint that returns all prices. In production environments, it is advisable to add error handling, timeouts, and retries to make the call more robust.
3. Deserializing the JSON response Most modern APIs respond in JSON. Calling response.json converts that text into native Python objects, typically a list of dictionaries containing the symbol and price fields among others.
4. Organizing data with pandas pandas transforms the list of dictionaries into a DataFrame, an in-memory tabular structure with indices and column labels that facilitates subsequent operations such as filtering, aggregation, and transformation for analysis or storage.
5. Filtering with boolean masking We use boolean masking to keep only the pairs we are interested in. The pattern price_df[price_df['symbol'].isin(TARGET_PAIRS)] evaluates a boolean mask and returns a new DataFrame with the rows whose symbol is in the TARGET_PAIRS list.
With these five steps, we have a basic but powerful ingestion. To take it to production, we recommend adding error handling with response.raise_for_status or try except, configuring timeouts in requests, instrumentation for logging and metrics, and persistence in databases or file formats such as parquet to store historical prices.
Automation: it can be run periodically with a scheduler such as cron or an orchestration service, thus building a time series of prices. For scaling and deployment, containers and cloud services such as AWS or Azure can be used.
About Q2BSTUDIO: we are a custom software and application development company specialized in artificial intelligence solutions, cybersecurity, and cloud services. We offer custom software, custom applications, and comprehensive artificial intelligence and AI solutions for businesses, including AI agents, Power BI integration, and business intelligence services. We also provide AWS and Azure cloud services and secure architectures to protect your data and operations.
Key services we offer: custom software development, custom applications, artificial intelligence consulting and deployment, cybersecurity detection and response, business intelligence services, Power BI integration, custom AI agents, and migration or management in AWS and Azure cloud services.
If you are looking to improve the value of your data with personalized solutions, automate ingestion pipelines, or apply artificial intelligence to your business processes, Q2BSTUDIO can help you design, implement, and operate the right solution, with a focus on security, scalability, and measurable results.
Keywords for positioning: custom applications, custom software, artificial intelligence, cybersecurity, AWS and Azure cloud services, business intelligence services, AI for businesses, AI agents, Power BI.
