Amazon S3 Python Connector
 

Amazon S3 Python Connector

SQL-based Access to Amazon S3 from Python

Easily connect Python-based Data Access, Visualization, ORM, ETL, AI/ML, and Custom Apps with Amazon S3!


  download   buy now


Other Amazon AWS Technologies

Other Amazon AWS Drivers


Amazon S3 Logo

Python Connector Libraries for Amazon S3 Data Connectivity. Integrate Amazon S3 with popular Python tools like Pandas, SQLAlchemy, Dash & petl. Easy-to-use Python Database API (DB-API) Modules connect Amazon S3 data with Python and any Python-based applications.


NOTE: To analyze data stored within S3 buckets, please refer to the CSV, JSON, XML, and Parquet Python Connectors. Alternatively, Amazon Athena can analyze data across multiple file-formats via AWS services.

Features

  • SQL access to Amazon S3 Buckets and Objects
  • Use SQL Stored Procedures to perform actions like uploading, downloading or copying objects
  • Connect to delimited files stored in S3 directly with the CSV & XML Drivers
  • Connect to live Amazon S3 data, for real-time data access
  • Full support for data aggregation and complex JOINs in SQL queries
  • Secure connectivity through modern cryptography, including TLS 1.2, SHA-256, ECC, etc.
  • Seamless integration with leading BI, reporting, and ETL tools and with custom applications

Specifications

  • Python Database API (DB-API) Modules for Amazon S3 .
  • Write SQL, get Amazon S3 data. Access Amazon S3 through standard Python Database Connectivity.
  • Integration with popular Python tools like Pandas, SQLAlchemy, Dash & petl.
  • Simple command-line based data exploration of Amazon S3 Buckets, Objects, and more!
  • Full Unicode support for data, parameter, & metadata.


CData Python Connectors in Action!

Watch the video overview for a first hand-look at the powerful data integration capabilities included in the CData Python Connectors.

WATCH THE PYTHON CONNECTOR VIDEO OVERVIEW

Python Connectivity with Amazon S3

Full-featured and consistent SQL access to any supported data source through Python


  • Universal Python Amazon S3 Connectivity

    Easily connect to Amazon S3 data from common Python-based frameworks, including:


    • Data Analysis/Visualization: Jupyter Notebook, pandas, Matplotlib
    • ORM: SQLAlchemy, SQLObject, Storm
    • Web Applications: Dash, Django
    • ETL: Apache Airflow, Luigi, Bonobo, Bubbles, petl
  • Popular Tooling Integration

    The Amazon S3 Connector integrates seamlessly with popular data science and developer tooling like Anaconda, Visual Studio Python IDE, PyCharm, and more. Real Python,

  • Replication and Caching

    Our replication and caching commands make it easy to copy data to local and cloud data stores such as Oracle, SQL Server, Google Cloud SQL, etc. The replication commands include many features that allow for intelligent incremental updates to cached data.

  • String, Date, Numeric SQL Functions

    The Amazon S3 Connector includes a library of 50 plus functions that can manipulate column values into the desired result. Popular examples include Regex, JSON, and XML processing functions.

  • Collaborative Query Processing

    Our Python Connector enhances the capabilities of Amazon S3 with additional client-side processing, when needed, to enable analytic summaries of data such as SUM, AVG, MAX, MIN, etc.

  • Easily Customizable and Configurable

    The data model exposed by our Amazon S3 Connector can easily be customized to add or remove tables/columns, change data types, etc. without requiring a new build. These customizations are supported at runtime using human-readable schema files that are easy to edit.

  • Enterprise-class Secure Connectivity

    Includes standard Enterprise-class security features such as TLS/ SSL data encryption for all client-server communications.

Connecting to Amazon S3 with Python

CData Python Connectors leverage the Database API (DB-API) interface to make it easy to work with Amazon S3 from a wide range of standard Python data tools. Connecting to and working with your data in Python follows a basic pattern, regardless of data source:

  • Configure the connection properties to Amazon S3
  • Query Amazon S3 to retrieve or update data
  • Connect your Amazon S3 data with Python data tools.


Connecting to Amazon S3 in Python

To connect to your data from Python, import the extension and create a connection:

import cdata.amazons3 as mod
conn = mod.connect("User=user@domain.com; Password=password;")

#Create cursor and iterate over results
cur = conn.cursor()
cur.execute("SELECT * FROM Buckets")
 
rs = cur.fetchall()
 
for row in rs:
print(row)

Once you import the extension, you can work with all of your enterprise data using the python modules and toolkits that you already know and love, quickly building apps that help you drive business.

Visualize Amazon S3 Data with pandas

The data-centric interfaces of the Amazon S3 Python Connector make it easy to integrate with popular tools like pandas and SQLAlchemy to visualize data in real-time. 

engine = create_engine("amazons3///Password=password&User=user")

df = pandas.read_sql("SELECT * FROM Buckets", engine)

df.plot()
plt.show()

Popular Python Videos:

Python Connectors, Jupyter Notebook, and pandas