Project 2: Comparing Bitcoin Exchanges using Cryptowatch’s API
(Due at 11:59pm on December 2, 2020)
Overview
For this project, we will compare the price and trading volume of Bitcoins across different exchanges.
The deadline for this project is Dec 2, 2020 (at 11:59pm).
Bitcoins exchanges act as market makers between buyers and sellers. It is not uncommon for different
exchanges to quote different prices at the same time. This does not necessarily lead to arbitrage opportunities
because of bid-ask spreads, fees, and other frictions.
In this project, we are interested in answering the following questions:
1. Do some exchanges consistently quote lower prices than others?
2. Which exchanges have the highest volume of bitcoin trading?
These are some of the questions investors should consider when deciding where to buy/sell bitcoins.
Note: For simplicity, we will only focus on prices and volume in this project. Of course, there are many other
important aspects to consider when choosing where to trade bitcoins. These include trading fees, minimum
trade size, and minimum initial investment.
This project has 3 parts and is worth 100 points in total:
1. Part 1: Understanding the Cryptowatch API (20 points)
a. Premiminaries
b. Questions
2. Part 2: Creating a Cryptowatch API (80 points)
a. Creating auxiliary functions
b. Obtaining a list of exchanges
c. Obtaining quotes
3. Part 3: Analysis (optional, 0 points)
Important:
This project is to be completed individually by each student. You should not collaborate. You
are allowed to post questions about the project in the thread. I will create in the Discussion
Board. However, please make sure you do not post partial solutions when asking questions, and
you do not answer any questions posted by other students. I will answer any project-related
questions myself.
Given the number of students that forgot about this important rule, I will say it again. Please:
1. Do NOT post partial solutions or tentative code when asking questions in the discussion board.
2. Do NOT answer any questions posted by other students regarding this project.
Many thanks in advance for your understanding.
1
Files
You received a zipped file with the following contents:
/
| _project2.py
| project2.pdf
|___files/
| |
| |
|__ _testfiles/
| | test.json
where represents your Zid and:
• represents the main folder containing all the project files. For example, if your Zid is z1234567,
this folder will be z1234567.
• _project2.py contains the functions you need to write for this project. Some of the functions
are already written, while others you will write yourself. Please see the instructions below for more
information.
• project2.pdf This file
• files/: This is the directory where the files you download from Cryptowatch will be saved. You must
include these files in your project submission
General Instructions
Important: Do not exchange complete or partial codes with other students.
Preparing the files for this project
1. Copy the main folder to your computer. Make sure it is inside a directory in the Python system
path (if you copy the folder into your PyCharm project folder, it will be). For example, if your
zid is z1234567, this folder will contain the following files:
z1234567/
| project2.pdf
| z1234567_project2.py
|
|___files/
| |
| |
|__ _testfiles/
| | ...
|... ...
2. Unless explicitly stated below, do not change any variable, import statement, function or
parameter names in the _project2.py module. In particular, do not change the
variables EXCH_LST, PERIOD, or SYMBOL, which will be described later.
3. Open the _project2.py module and change the variable PRJDIR, so it points to the location of
the folder. In the example above, this variable should point to the location of the z1234567
folder.
4. Make sure the sub-folder called files is empty for now. During the project, you will download a series
of files and save them inside this folder. Do not include other files inside this folder. All files inside this
folder will be part of your submission and we will use them when assessing your project.
2
Instructions for Part 1
1. Read the Part 1 section below. Make sure you answer all the questions in the module
_project2.py. Do not submit answers in any other format. We will parse your completed
_project2.py file for the answers of both Parts 1 and 2.
2. I cannot emphasize this enough: Although the questions for Part 1 are included in this document, you
should provide the answers in the _project2.py file!
Instructions for Part 2
1. Follow the instructions to complete the remaining functions in the _project2.py module.
2. IMPORTANT After you are done, run the function main without any parameters (which will download
the exchanges and OHLC files again). This will make sure the files folder contains the following files:
• exchanges.json
• One file for each exchange in EXCH_LST and the pair in SYMBOL These are the files that will be
assessed. Other files in that folder will not be considered.
Instructions for Part 3
Part 3 is optional and will not be marked (no bonus marks either).
Project submission
You should submit a zipped file containing the completed _project.py module and all the
files in files/. The easiest way to do that is to compress the entire folder into a zip file
once your are done with the project. This is the zip file you must submit
Part 1: Understanding the Cryptowatch API
During the first part of this project, you will familiarize yourself with Cryptowatch’s API and the data we
will use in our analysis.
Preliminaries:
Cryptowatch is a trading platform that provides real-time market data for many cryptocurrency exchanges.
The real-time data on Cryptowatch is provided directly from cryptocurrency exchanges.
Cryptowatch allow users to access their data through a RESTful API. Free/anonymous access is limited,
however.
You can find more information about Cryptowatch’s API by reading the Documentation.
OHLC Quotes
In this project, we will collect open, high, low, and close prices for bitcoins trading in different exchanges. We
often refer to these quotes collectively as OHLC, which stands for Open, High, Low, and Close. The OHLC
quotes are always associated with a period of time so that:
• Open: The quote at the beginning of the period
• High: The highest quote during the period
• Low: The lowest quote during the period
• Close: The quote at the end of the period
For instance, suppose we set the period to be “5 minutes”. The OHLC quotes for a five-minute period starting
at midnight today would be obtained as follows:
3
1. Split the day into 5-minute intervals:
first second third ....
----|---------|---------|-------|--------|--> Time
12:00am 12:05am 12:10am ..... now
2. Compute the open, high, low, and close prices for each interval
3. Return the data in some pre-specified data format, for instance:
row timestamp open high low close
1 12:05 00:00:00 .. .. .. ..
2 12:10 00:00:00 .. .. .. ..
.. .. .. .. .. ..
The timestamp above refers to the end of the interval. This means that the first interval (the first data
point) will start at 12:00am and end at 12:05am. The OHLC quotes for the first interval (row=1) would then
represent1
:
• Open: The first quote immediately after 12:00am
• High: The highest quote from 12:00am to 12:05am
• Low: The lowest quote from 12:00am to 12:05am
• Close: The last quote before (or at) 12:05am
• First interval is (12:00am, 12:05am]
• Second interval is (12:05am, 12:10am]
This means that: - A quote at 12:00 00:00:01 would belong to the first interval - A quote at 12:05
00:00:00 would belong to the first interval - A quote at 12:05 00:00:01 would belong to the second
interval
In the context of OHLC quotes, each individual interval is sometimes referred to as a candle. This comes
from the popular candlestick charts used to represent trading patterns. For example, the table above contains
the “5-minute” candles for the previous 24-hours, where each row is a candle.
Unix Timestamps
A UNIX timestamp is an integer representing a point in time. The value of a UNIX timestamp represents the
number of seconds that have elapsed since some arbitrary date and time (taken to be midnight of January 1,
1970, UTC), corrected for leap seconds.
Cryptowatch uses these timestamps to represent the date/times that define OHLC intervals. Cryptowatch
returns dates in UTC. You can use the datetime package to convert between these timestamps and local
or UTC date/time. The relevant methods are fromtimestamp and utcfromtimestamp. If ts is an integer
representing a Unix timestamp, these methods will return datetime instances so that:
• .fromtimestamp(ts) --> datetime object (for local date/time)
• .utcfromtimestamp(ts) --> datetime object (for UTC date/time)
For instance, consider the unix timestamp 1474736400. What is the UTC and local datetime for this
timestamp?
import datetime as dt
ts = 1474736400
1A valid question at this point would be: “In the example above, does the timestamp 12:05 00:00:00 belong to the first or
second interval?” For this project, we will assume semi-open intervals (open at the beginning of the interval, closed at the end of
the interval). For instance, in the example above:
4
# This is the local date for this timestamp
local_dt = dt.datetime.fromtimestamp(ts)
print(local_dt) # returns '2016-09-25 03:00:00'
# This is the UTC date for this timestamp
# (note that the method is "utcfromtimestap")
utc_dt = dt.datetime.utcfromtimestamp(ts)
print(utc_dt) # returns '2016-09-24 17:00:00'
Questions (20 points)
Please answer the questions below in the appropriate section of your _project2.py module.
This section is clearly defined in that module using the label “ANSWERS TO PART 1
QUESTIONS”
For each topic below, read the relevant documentation and answer the following questions.
1. Rate limit:
Assume all questions below refer to free/anonymous access to Cryptowatch’s API.
• Q1: Do you need an API key for free/anonymous access?
• Q2: What is the initial value of your allowance in Cryptowatch credits?
• Q3: Do all requests cost the same in credits?
2. Assets
• Q4: What an asset?
– (one sentence, max 150 characters)
• Q5: What is the symbol for the Bitcoin asset?
3. Pairs
• Q6: What is a pair?
– (one sentence, max 150 characters)
• Q7: What is the symbol for the USD price of one Bitcoin (i.e., the Bitcoin, USD pair)?
4. Markets
• Q8: What is a market?
– (one sentence, max 150 characters)
• Q9: What is the market endpoint URL for the pair “btcusd” and exchange “kraken”? Hint: make
sure you use the pair btcusd!
– (one url, max 150 characters)
• Use any web browser to “navigate” to this endpoint. Given the raw data contained in the response,
answer the following questions:
– Q10: In which format is the raw data represented?
– Q11: Based on the data contained in the response, is this market active? In other words, can
you use USD to trade bitcoins in this exchange?
– Q12: Based on the data contained in the response, what is the id for this market? This is the
integer Cryptowatch uses to identify this market.
5. OHLC Candlesticks
• Q13: What is the endpoint URL for the OHLC market represented by the Kraken exchange and
the btcusd pair? Assume that you do not have any optional query parameters.
• Q14: For the resources returned by OHLC endpoints (without query parameters), what is the
length of the interval for the candles stored under the JSON key “14400”? Hint: The page OHLC
5
Candlesticks contains an example of a response to a request with an endpoint similar to the one
above. You can use this response and the period table at the end of OHLC Candlesticks to answer
this question.
• Q15: Suppose you are given the following candle in a string:
[
1474736400,
8744,
8756.1,
8710,
8753.5,
91.58314308,
799449.488966417
]
Which of these values correspond to the close quote?
• Q16: In the candle above, the unix timestamp is 1474736400. Use the datetime module to answer
the following question: What is the UTC date and time corresponding to this timestamp?
Part 2
During this part of the project, you will modify some functions in the _project2.py module you
received.
Important:
As discussed above, Cryptowatch allows you to retrieve a certain amount of data for free for every 24-hour
period. This amount is more than enough to complete this project inside a 24-hour period (it corresponds to
hundreds of API requests). However, Make sure you do not make unnecessary requests and run out of credits!
Making sure you do not run out of credit is part of the assessment for this project. You will
not get an extension if you run out of credits.
Setup
All that is required is changing the value of the variable PRJDIR in the _project2.py so it points to
the location where you extracted the Zid folder included in the zip file you received. This variable should end
with your Zid. For instance, if your Zid is z1234567 the value of PRJDIR should be:
# Linux/Mac
PRJDIR = '
/z1234567'
# Windows
PRJDIR = r'
\z1234567'
If your PRJDIR variable is not set correctly, the module will produce an error the first time you try to run it.
Important
Please do not modify any of the following: - Constants (the variables in all caps) with the exception of
PRJDIR (and the questions for Part 1). - The name or parameters of any function. - Any function not listed
in the outline below.
Outline
Note: The module you were given contains many test functions that you can use (they all start with _test.
You can change these functions if you want. We have not given you the output of the _test functions for
6
obvious reasons.
Below is the suggested outline you should follow to complete this part of the project:
1. Complete all the auxiliary functions
• to_json: converts a Python object to a JSON string and save it to a file. Follow the instructions
in the docstring. You can use the function _test_to_json to test this function.
• from_json: Reads a file containing a JSON string and convert it to an object. Follow the
instructions in the docstring. You can use the function _test_from_json if you want.
• get_ohlc_loc: This function is already written and should not be modified.
2. Download and save a file containing information on all exchanges in Cryptowatch
• write_exchange: Follow the instructions in the docstring.
• read_exchanges: Read the contents of the file saved by write_exchanges and returns a dataframe
with the result.
• Note: The reason for this two-step approach is to minimize the number of API calls. You only
have to run the function write_exchange once! After you have downloaded the list of exchanges,
you should not call this function anymore. Instead, use read_exchanges.
3. Create a function to download OHLC quotes for a given market
• write_ohlc: Follow the instructions in the docstring.
• Call the _test_write_ohlc function and examine the contents of the file you downloaded. Once you
are finishing debugging, do not call this function again (remember your credits are limited).
• Open the file ...files/kraken_btceur.json and examine its contents. It should be similar to what
is described in the write_ohlc docstring.
1. Create a function to produce a dictionary with the contents of a candle.
• candle_to_dic: follow the instructions in the docstring.
• You can use the function _test_candle_to_dic to test it.
1. Create a function to return the number of seconds in a given period label.
• period_to_secs: follow the instructions in the docstring. Keep it simple.
1. Create a function to read the quotes for a given market and period.
• read_ohlc: follow the instructions in the docstring.
• You can use the _test_read_ohlc function to test it.
1. Create a function to read the quotes for a given market and period and return a dataframe.
• mk_ohlc_df: follow the instructions in the docstring.
• You can use the _test_mk_ohlc_df function to test it.
1. Create a function to return a list of exchange symbols
• get_valid_exchanges: follow instructions in the docstring
2. Download the relevant data
• get_all_data: follow the instructions in the docstring
• Note: make sure you do not modify the value of the constants SYMBOL PERIOD and EXCH_LST.
Part 3
This part will not be marked. It is optional but it will give you an idea of how this API can be used.
1. Uncomment and run the function compare_exchanges
2. Do you find that some exchange has consistently lower quotes?
3. Do some research and try to figure out why some exchanges have lower prices than others.