LOG IN
Find MentorsFind FreelancersCommunity PostBlog
SIGN UP
LOG IN
Codementor Events
Siddhant Singh
Follow
Python & JS Full Stack Developer (MERN & Django)

Using Python & Apscheduler to retrieve data from Venmo API(Multiple Pages) to Csv files periodically(Custom Time)

Published Dec 20, 2017
Using Python & Apscheduler to retrieve data from Venmo API(Multiple Pages) to Csv files periodically(Custom Time)

Getting Data From the Venmo Api

I have used Requests module of Python to retrieve data from Venmo API--https://venmo.com/api/v5/public

import json
import csv
import requests


r = requests.get("https://venmo.com/api/v5/public")
print(r.status_code)
print(r.encoding)
print(r.headers['content-type'])

data = r.json()
print(data.keys())

Thia json data has two keys = Paging and the main Data

[
 {
  "name": "stdout",
  "output_type": "stream",
  "text": "200\nNone\napplication/json\ndict_keys(['paging', 'data'])\n"
 }
]

print(data["data"])

Main data format from Venmo API

[
 {
  "name": "stdout",
  "output_type": "stream",
  "text": "[{'payment_id': 735051441, 'permalink': '/story/5a3a6e995a877f6e3827e00d', 'via': '', 'action_links': {}, 'story_id': '5a3a6e995a877f6e3827e00d', 'comments': [], 'updated_time': '2017-12-20T14:07:21Z', 'audience': 'public', 'actor': {'username': 'John-Swatkowski', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'John Swatkowski', 'firstname': 'John', 'lastname': 'Swatkowski', 'cancelled': False, 'date_created': '2016-10-10T02:14:17', 'external_id': '2057705887891456079', 'id': '14651665'}, 'transactions': [{'target': {'username': 'Shelton-Sekac', 'picture': 'https://venmopics.appspot.com/u/v1/n/829cd1e9-28cc-4478-86f6-3f0b5aeeba7b', 'is_business': False, 'name': 'Shelton Sekac', 'firstname': 'Shelton', 'lastname': 'Sekac', 'cancelled': False, 'date_created': '2016-11-01T15:20:52', 'external_id': '2074046854332416210', 'id': '15289044'}}], 'created_time': '2017-12-20T14:07:21Z', 'mentions': [], 'message': 'You get your money back for third.  Fantasy Football 2017', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051442, 'permalink': '/story/5a3a6e995a877f6e3827e115', 'via': '', 'action_links': {}, 'story_id': '5a3a6e995a877f6e3827e115', 'comments': [], 'updated_time': '2017-12-20T14:07:21Z', 'audience': 'public', 'actor': {'username': 'adean1024', 'picture': 'https://venmopics.appspot.com/u/v1/m/aaa8daac-41bb-43d8-8feb-7f7d17969356', 'is_business': False, 'name': 'Andy Dean', 'firstname': 'Andy', 'lastname': 'Dean', 'cancelled': False, 'date_created': '2014-11-17T15:51:33', 'external_id': '1555847649951744766', 'id': '2853641'}, 'transactions': [{'target': {'username': 'Sean-McGonagle', 'picture': 'https://graph.facebook.com/v2.10/1138394506/picture?type=large', 'is_business': False, 'name': 'Sean McGonagle', 'firstname': 'Sean', 'lastname': 'McGonagle', 'cancelled': False, 'date_created': '2014-07-29T15:57:26', 'external_id': '1475400504967168117', 'id': '1944009'}}], 'created_time': '2017-12-20T14:07:21Z', 'mentions': [], 'message': 'Chelsea W', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051446, 'permalink': '/story/5a3a6e995a877f6e3827e5d0', 'via': '', 'action_links': {}, 'story_id': '5a3a6e995a877f6e3827e5d0', 'comments': [], 'updated_time': '2017-12-20T14:07:21Z', 'audience': 'public', 'actor': {'username': 'Jennie-Edwards-1', 'picture': 'https://venmopics.appspot.com/u/v2/n/8147d71d-277d-4842-8ddf-96c859a8e2de', 'is_business': False, 'name': 'Jennie Edwards', 'firstname': 'Jennie', 'lastname': 'Edwards', 'cancelled': False, 'date_created': '2014-05-09T18:33:28', 'external_id': '1416772204888064804', 'id': '1461315'}, 'transactions': [{'target': {'username': 'Emily-Harvey-4', 'picture': 'https://venmopics.appspot.com/u/v1/n/f376ba37-88e6-4739-b02a-bcf7bb342d1b', 'is_business': False, 'name': 'Emily Harvey', 'firstname': 'Emily', 'lastname': 'Harvey', 'cancelled': False, 'date_created': '2014-10-06T13:12:23', 'external_id': '1525326958034944935', 'id': '2518841'}}], 'created_time': '2017-12-20T14:07:21Z', 'mentions': [], 'message': '\ud83d\udc1e\ud83d\udc1c', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051447, 'permalink': '/story/5a3a6e995a877f6e3827e723', 'via': '', 'action_links': {}, 'story_id': '5a3a6e995a877f6e3827e723', 'comments': [], 'updated_time': '2017-12-20T14:07:21Z', 'audience': 'public', 'actor': {'username': 'RondaThomas', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'Ronda Thomas', 'firstname': 'Ronda', 'lastname': 'Thomas', 'cancelled': False, 'date_created': '2016-11-15T22:02:29', 'external_id': '2084395854856192970', 'id': '15680443'}, 'transactions': [{'target': {'username': 'Jason-Rouse-7', 'picture': 'https://graph.facebook.com/v2.10/10155621277804342/picture?type=large', 'is_business': False, 'name': 'Jason Rouse', 'firstname': 'Jason', 'lastname': 'Rouse', 'cancelled': False, 'date_created': '2017-12-20T01:43:32', 'external_id': '2373692629712897023', 'id': '30192464'}}], 'created_time': '2017-12-20T14:07:21Z', 'mentions': [], 'message': 'Food and service!', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051449, 'permalink': '/story/5a3a6e995a877f6e3827ed6c', 'via': '', 'action_links': {}, 'story_id': '5a3a6e995a877f6e3827ed6c', 'comments': [], 'updated_time': '2017-12-20T14:07:21Z', 'audience': 'public', 'actor': {'username': 'Mackenziesmith9', 'picture': 'https://venmopics.appspot.com/u/v1/n/d30805c2-7b57-4abf-91b2-f85e20025525', 'is_business': False, 'name': 'Mackenzie Smith', 'firstname': 'Mackenzie', 'lastname': 'Smith', 'cancelled': False, 'date_created': '2017-10-12T09:41:38', 'external_id': '2323923739869184901', 'id': '27331094'}, 'transactions': [{'target': {'username': 'Amy-Allaire-1', 'picture': 'https://venmopics.appspot.com/u/v1/m/ba844d79-4cf5-4f32-9242-991cdbfb721a', 'is_business': False, 'name': 'Amy Allaire', 'firstname': 'Amy', 'lastname': 'Allaire', 'cancelled': False, 'date_created': '2017-09-01T20:33:05', 'external_id': '2294535820410880144', 'id': '25514859'}}], 'created_time': '2017-12-20T14:07:21Z', 'mentions': [], 'message': '\ud83c\udfe0\ud83c\udfe0\ud83c\udfe0\ud83c\udfe0\ud83c\udfe0', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051450, 'permalink': '/story/5a3a6e9a5a877f6e3827ee00', 'via': '', 'action_links': {}, 'story_id': '5a3a6e9a5a877f6e3827ee00', 'comments': [], 'updated_time': '2017-12-20T14:07:21Z', 'audience': 'public', 'actor': {'username': 'Brian-Swanson-4', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'Brian Swanson', 'firstname': 'Brian', 'lastname': 'Swanson', 'cancelled': False, 'date_created': '2014-05-11T06:04:59', 'external_id': '1417845032353792066', 'id': '1469653'}, 'transactions': [{'target': {'username': 'Mackenzie-Trubee', 'picture': 'https://venmopics.appspot.com/u/v1/m/da97d15e-798d-463a-9d0d-1a04abd95270', 'is_business': False, 'name': 'Mackenzie Trubee', 'firstname': 'Mackenzie', 'lastname': 'Trubee', 'cancelled': False, 'date_created': '2014-06-27T22:43:44', 'external_id': '1452412179054592767', 'id': '1749033'}}], 'created_time': '2017-12-20T14:07:21Z', 'mentions': [], 'message': '15x5', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051431, 'permalink': '/story/5a3a6e985a877f6e3827cbe7', 'via': '', 'action_links': {}, 'story_id': '5a3a6e985a877f6e3827cbe7', 'comments': [], 'updated_time': '2017-12-20T14:07:20Z', 'audience': 'public', 'actor': {'username': 'lrfarber', 'picture': 'https://venmopics.appspot.com/u/v3/n/8fecb8dc-2c40-4b75-9502-151dc2217272', 'is_business': False, 'name': 'Leyah Farber', 'firstname': 'Leyah', 'lastname': 'Farber', 'cancelled': False, 'date_created': '2014-02-23T21:01:26', 'external_id': '1362488499109888287', 'id': '1082483'}, 'transactions': [{'target': {'username': 'Alex-Colcord', 'picture': 'https://venmopics.appspot.com/u/v1/n/0bf2ee2e-bf4e-4505-bcb4-de75fe4d7488', 'is_business': False, 'name': 'Alex Colcord', 'firstname': 'Alex', 'lastname': 'Colcord', 'cancelled': False, 'date_created': '2014-12-19T22:06:50', 'external_id': '1579229359636480453', 'id': '3135671'}}], 'created_time': '2017-12-20T14:07:20Z', 'mentions': [], 'message': '\ud83d\udc55\ud83d\udc54\ud83d\udc56', 'type': 'charge', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051435, 'permalink': '/story/5a3a6e985a877f6e3827d3d1', 'via': '', 'action_links': {}, 'story_id': '5a3a6e985a877f6e3827d3d1', 'comments': [], 'updated_time': '2017-12-20T14:07:20Z', 'audience': 'public', 'actor': {'username': 'mverkinderen', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'Matthew Verkinderen', 'firstname': 'Matthew', 'lastname': 'Verkinderen', 'cancelled': False, 'date_created': '2015-07-26T17:51:27', 'external_id': '1737826706128896417', 'id': '5593975'}, 'transactions': [{'target': {'username': 'mikemillsm1', 'picture': 'https://venmopics.appspot.com/u/v1/n/f7f66188-eeda-4c5c-9ba0-df132f6533cb', 'is_business': False, 'name': 'Mike Mills', 'firstname': 'Mike', 'lastname': 'Mills', 'cancelled': False, 'date_created': '2017-01-29T00:21:42', 'external_id': '2138099329007616854', 'id': '17831545'}}], 'created_time': '2017-12-20T14:07:20Z', 'mentions': [], 'message': ':pumpkin_spice_latte:\u2615:pumpkin_spice_latte:', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051436, 'permalink': '/story/5a3a6e985a877f6e3827d586', 'via': '', 'action_links': {}, 'story_id': '5a3a6e985a877f6e3827d586', 'comments': [], 'updated_time': '2017-12-20T14:07:20Z', 'audience': 'public', 'actor': {'username': 'zoelibert', 'picture': 'https://venmopics.appspot.com/u/v4/n/5ee83483-b26e-4d09-8b7d-663d52712503', 'is_business': False, 'name': 'Zoe Libert', 'firstname': 'Zoe', 'lastname': 'Libert', 'cancelled': False, 'date_created': '2015-11-01T23:38:20', 'external_id': '1809029319884800779', 'id': '7175761'}, 'transactions': [{'target': {'username': 'Joseph-Nelson-9', 'picture': 'https://venmopics.appspot.com/u/v1/n/e2ced4cd-67ad-4370-b489-7337cb5c6fb2', 'is_business': False, 'name': 'Joseph Nelson', 'firstname': 'Joseph', 'lastname': 'Nelson', 'cancelled': False, 'date_created': '2015-02-11T22:21:59', 'external_id': '1618374874365952205', 'id': '3660456'}}], 'created_time': '2017-12-20T14:07:20Z', 'mentions': [], 'message': '\ud83d\udcaf\ud83d\udcaf', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051422, 'permalink': '/story/5a3a6e975a877f6e3827be74', 'via': '', 'action_links': {}, 'story_id': '5a3a6e975a877f6e3827be74', 'comments': [], 'updated_time': '2017-12-20T14:07:19Z', 'audience': 'public', 'actor': {'username': 'Jamison-Kinnane', 'picture': 'https://venmopics.appspot.com/u/v1/n/bf002391-e1e0-4934-b467-d27f4b721f10', 'is_business': False, 'name': 'Jamison Kinnane', 'firstname': 'Jamison', 'lastname': 'Kinnane', 'cancelled': False, 'date_created': '2014-06-30T15:03:38', 'external_id': '1454354930335744667', 'id': '1762830'}, 'transactions': [{'target': {'username': 'Nally-Kinnane', 'picture': 'https://venmopics.appspot.com/u/v3/m/54225fca-4e0a-4ffc-9afa-838b776243f7', 'is_business': False, 'name': 'Nally Kinnane', 'firstname': 'Nally', 'lastname': 'Kinnane', 'cancelled': False, 'date_created': '2014-02-01T15:15:40', 'external_id': '1346369402961920090', 'id': '980066'}}], 'created_time': '2017-12-20T14:07:19Z', 'mentions': [], 'message': 'More p', 'type': 'charge', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051424, 'permalink': '/story/5a3a6e975a877f6e3827c2f7', 'via': '', 'action_links': {}, 'story_id': '5a3a6e975a877f6e3827c2f7', 'comments': [], 'updated_time': '2017-12-20T14:07:19Z', 'audience': 'public', 'actor': {'username': 'Joel-Boivin', 'picture': 'https://venmopics.appspot.com/u/v1/m/da0d31b0-9c92-4020-8102-1900cd98d66d', 'is_business': False, 'name': 'Joel Boivin', 'firstname': 'Joel', 'lastname': 'Boivin', 'cancelled': False, 'date_created': '2017-04-21T22:21:46', 'external_id': '2198195350274048061', 'id': '20496080'}, 'transactions': [{'target': {'username': 'Matthew-Bessette-3', 'picture': 'https://venmopics.appspot.com/u/v1/m/fcb7a8c4-f8e1-44d8-98e6-103990de6079', 'is_business': False, 'name': 'Matthew Bessette', 'firstname': 'Matthew', 'lastname': 'Bessette', 'cancelled': False, 'date_created': '2017-04-21T22:15:46', 'external_id': '2198192330375168955', 'id': '20495878'}}], 'created_time': '2017-12-20T14:07:19Z', 'mentions': [], 'message': '\ud83e\udd5c', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051426, 'permalink': '/story/5a3a6e975a877f6e3827c491', 'via': '', 'action_links': {}, 'story_id': '5a3a6e975a877f6e3827c491', 'comments': [], 'updated_time': '2017-12-20T14:07:19Z', 'audience': 'public', 'actor': {'username': 'Cassidy-Braun', 'picture': 'https://venmopics.appspot.com/u/v1/m/d7846df2-7fe0-43ed-9f86-593ddeb11034', 'is_business': False, 'name': 'Cassidy Braun', 'firstname': 'Cassidy', 'lastname': 'Braun', 'cancelled': False, 'date_created': '2016-01-09T22:52:32', 'external_id': '1859015793442816360', 'id': '8273477'}, 'transactions': [{'target': {'username': 'NikkiScalia', 'picture': 'https://venmopics.appspot.com/u/v2/n/e90e1d19-d53d-47ae-bc3e-7b8930dbeb41', 'is_business': False, 'name': 'Nikki Scalia', 'firstname': 'Nikki', 'lastname': 'Scalia', 'cancelled': False, 'date_created': '2015-09-09T20:58:20', 'external_id': '1770535675494400436', 'id': '6311954'}}], 'created_time': '2017-12-20T14:07:19Z', 'mentions': [], 'message': 'Good thing Ty doesn\u2019t have :venmo:', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051429, 'permalink': '/story/5a3a6e975a877f6e3827cb01', 'via': '', 'action_links': {}, 'story_id': '5a3a6e975a877f6e3827cb01', 'comments': [], 'updated_time': '2017-12-20T14:07:19Z', 'audience': 'public', 'actor': {'username': 'Megan-Heim', 'picture': 'https://venmopics.appspot.com/u/v2/m/63b228d7-7df1-48ed-b104-f70cc28a67a7', 'is_business': False, 'name': 'Megan Heim', 'firstname': 'Megan', 'lastname': 'Heim', 'cancelled': False, 'date_created': '2014-10-14T23:43:57', 'external_id': '1531443041796096466', 'id': '2585756'}, 'transactions': [{'target': {'username': 'Naomi-Scher', 'picture': 'https://venmopics.appspot.com/u/v3/n/ef397a91-2dd9-4eb4-9d44-253daac57905', 'is_business': False, 'name': 'Naomi Scher', 'firstname': 'Naomi', 'lastname': 'Scher', 'cancelled': False, 'date_created': '2015-02-27T18:30:45', 'external_id': '1629854902517760574', 'id': '3820896'}}], 'created_time': '2017-12-20T14:07:19Z', 'mentions': [], 'message': 'Loki thanks his third mom for dog walking funds', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051411, 'permalink': '/story/5a3a6e965a877f6e3827a6a3', 'via': '', 'action_links': {}, 'story_id': '5a3a6e965a877f6e3827a6a3', 'comments': [], 'updated_time': '2017-12-20T14:07:18Z', 'audience': 'public', 'actor': {'username': 'Michaela-Klym', 'picture': 'https://venmopics.appspot.com/u/v1/n/f30cdf49-db8a-495a-8490-f7eec554ec23', 'is_business': False, 'name': 'Michaela Klym', 'firstname': 'Michaela', 'lastname': 'Klym', 'cancelled': False, 'date_created': '2014-09-09T02:17:19', 'external_id': '1505428307443712255', 'id': '2298425'}, 'transactions': [{'target': {'username': 'Noble-Stultz', 'picture': 'https://graph.facebook.com/v2.10/1024036256/picture?type=large', 'is_business': False, 'name': 'Noble Stultz', 'firstname': 'Noble', 'lastname': 'Stultz', 'cancelled': False, 'date_created': '2013-10-27T23:39:11', 'external_id': '1276319585271808439', 'id': '651693'}}], 'created_time': '2017-12-20T14:07:18Z', 'mentions': [], 'message': '\ud83d\ude97', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051414, 'permalink': '/story/5a3a6e965a877f6e3827aaa0', 'via': '', 'action_links': {}, 'story_id': '5a3a6e965a877f6e3827aaa0', 'comments': [], 'updated_time': '2017-12-20T14:07:18Z', 'audience': 'public', 'actor': {'username': 'KellyMarie_92', 'picture': 'https://venmopics.appspot.com/u/v1/m/6eb1a3b1-55c8-45ee-b7e9-b04360658d75', 'is_business': False, 'name': 'Kelly Marie Castro', 'firstname': 'Kelly Marie', 'lastname': 'Castro', 'cancelled': False, 'date_created': '2014-11-20T00:06:03', 'external_id': '1557546091413504270', 'id': '2876270'}, 'transactions': [{'target': {'username': 'spooling', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'Stefano Pulignano', 'firstname': 'Stefano', 'lastname': 'Pulignano', 'cancelled': False, 'date_created': '2016-02-05T17:21:54', 'external_id': '1878418324979712686', 'id': '8765586'}}], 'created_time': '2017-12-20T14:07:18Z', 'mentions': [], 'message': 'Let\u2019s help turn Type One to Type None! \ud83d\ude0a\\nI\u2019m collecting donations to the Juvenile Diabetes Research Foundation for my 25th birthday! \ud83d\udc99\\nCheck out my birthday fundraiser on Facebook for more info, or to make a larger donation! \ud83c\udf89\\n~Any amount is appreciated~ \u263a\ufe0f\\n((I\u2019ll be donating all $ collected via Venmo to my fundraiser))', 'type': 'charge', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051415, 'permalink': '/story/5a3a6e965a877f6e3827ac53', 'via': '', 'action_links': {}, 'story_id': '5a3a6e965a877f6e3827ac53', 'comments': [], 'updated_time': '2017-12-20T14:07:18Z', 'audience': 'public', 'actor': {'username': 'Aaron-Orr-7', 'picture': 'https://venmopics.appspot.com/u/v1/m/0e104ec6-aa14-471f-b945-ef46f86de9bb', 'is_business': False, 'name': 'Aaron Orr', 'firstname': 'Aaron', 'lastname': 'Orr', 'cancelled': False, 'date_created': '2017-12-20T05:23:18', 'external_id': '2373803241897984043', 'id': '30201116'}, 'transactions': [{'target': {'username': 'Ammon-Anderson', 'picture': 'https://venmopics.appspot.com/u/v1/m/83df2895-9ded-4eb7-8cfb-e68b9ee91795', 'is_business': False, 'name': 'Ammon Anderson', 'firstname': 'Ammon', 'lastname': 'Anderson', 'cancelled': False, 'date_created': '2016-12-21T15:59:42', 'external_id': '2110305186349056599', 'id': '16717946'}}], 'created_time': '2017-12-20T14:07:18Z', 'mentions': [], 'message': 'Brotherly love ;)', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051418, 'permalink': '/story/5a3a6e965a877f6e3827b117', 'via': '', 'action_links': {}, 'story_id': '5a3a6e965a877f6e3827b117', 'comments': [], 'updated_time': '2017-12-20T14:07:18Z', 'audience': 'public', 'actor': {'username': 'Tim-Spicer', 'picture': 'https://venmopics.appspot.com/u/v1/n/35c81b05-18ed-4e88-aab5-89a90245f839', 'is_business': False, 'name': 'Tim Spicer', 'firstname': 'Tim', 'lastname': 'Spicer', 'cancelled': False, 'date_created': '2016-03-27T06:03:18', 'external_id': '1915040336707584977', 'id': '9744909'}, 'transactions': [{'target': {'username': 'Shaquita-Toombs', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'Shaquita Toombs', 'firstname': 'Shaquita', 'lastname': 'Toombs', 'cancelled': False, 'date_created': '2017-05-18T00:52:46', 'external_id': '2217115520073728993', 'id': '21394360'}}], 'created_time': '2017-12-20T14:07:18Z', 'mentions': [], 'message': 'Love', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051420, 'permalink': '/story/5a3a6e965a877f6e3827b62c', 'via': '', 'action_links': {}, 'story_id': '5a3a6e965a877f6e3827b62c', 'comments': [], 'updated_time': '2017-12-20T14:07:18Z', 'audience': 'public', 'actor': {'username': 'ashlee-nava', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'ashlee nava', 'firstname': 'ashlee', 'lastname': 'nava', 'cancelled': False, 'date_created': '2016-01-05T14:15:27', 'external_id': '1855856433954816337', 'id': '8189383'}, 'transactions': [{'target': 'an email'}], 'created_time': '2017-12-20T14:07:18Z', 'mentions': [], 'message': '\ud83c\udf77 \ud83c\udf77 thank you!', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051421, 'permalink': '/story/5a3a6e965a877f6e3827b671', 'via': '', 'action_links': {}, 'story_id': '5a3a6e965a877f6e3827b671', 'comments': [], 'updated_time': '2017-12-20T14:07:18Z', 'audience': 'public', 'actor': {'username': 'Samanthablackk', 'picture': 'https://venmopics.appspot.com/u/v1/n/e1e42586-3c39-4ac5-bc5f-b79338daf024', 'is_business': False, 'name': 'Samantha Black', 'firstname': 'Samantha', 'lastname': 'Black', 'cancelled': False, 'date_created': '2016-07-31T19:03:59', 'external_id': '2006755009626112519', 'id': '12542633'}, 'transactions': [{'target': {'username': 'Arrie-Claesson', 'picture': 'https://s3.amazonaws.com/venmo/no-image.gif', 'is_business': False, 'name': 'Arrie Claesson', 'firstname': 'Arrie', 'lastname': 'Claesson', 'cancelled': False, 'date_created': '2015-03-22T16:03:53', 'external_id': '1646450823921664897', 'id': '4077250'}}], 'created_time': '2017-12-20T14:07:18Z', 'mentions': [], 'message': 'Jk I love you \ud83d\ude18', 'type': 'payment', 'likes': {'count': 0, 'data': []}}, {'payment_id': 735051397, 'permalink': '/story/5a3a6e955a877f6e3827945d', 'via': '', 'action_links': {}, 'story_id': '5a3a6e955a877f6e3827945d', 'comments': [], 'updated_time': '2017-12-20T14:07:17Z', 'audience': 'public', 'actor': {'username': 'Hannah-Loftis', 'picture': 'https://venmopics.appspot.com/u/v1/m/e1a1acb9-783b-4265-9c1a-0a63cb7d888c', 'is_business': False, 'name': 'Hannah Loftis', 'firstname': 'Hannah', 'lastname': 'Loftis', 'cancelled': False, 'date_created': '2016-12-09T19:52:57', 'external_id': '2101725276143616552', 'id': '16359875'}, 'transactions': [{'target': {'username': 'Lora-Loftis-1', 'picture': 'https://venmopics.appspot.com/u/v1/m/b472e593-febc-4e19-b3b3-d740580548fe', 'is_business': False, 'name': 'Lora Loftis', 'firstname': 'Lora', 'lastname': 'Loftis', 'cancelled': False, 'date_created': '2017-12-01T21:47:56', 'external_id': '2360528085188608341', 'id': '29374792'}}], 'created_time': '2017-12-20T14:07:17Z', 'mentions': [], 'message': 'Chiropractor', 'type': 'charge', 'likes': {'count': 0, 'data': []}}]\n"
 }
]

print(data["paging"]["next"])

URL for the next page to retrieve data

[
 {
  "name": "stdout",
  "output_type": "stream",
  "text": "https://venmo.com/api/v5/public?until=1513778837\n"
 }
]

Here I have made a function to recursively query data from the given url and saves it to a csv file. Filename will be taken from the given url endpoints for the ease of understanding and handling. ex. public?until=1513778837.csv

# making a function for it
def api_to_csv(url):
    r = requests.get(url)
    # data in json format
    data = r.json()
    
    public_data = data['data']

    lol = url.split("/")
    
    # give unique csv names to each page from API
    
    filename = lol[-1] + ".csv"
    
    with open(filename, 'w') as main_data:
        # create the csv writer object

        csvwriter = csv.writer(main_data)

        count = 0

        for temp in public_data:

              if count == 0:

                     header = temp.keys()

                     csvwriter.writerow(header)

                     count += 1

              csvwriter.writerow(temp.values())



        url = data["paging"]["next"]
        if url == "":
            pass


        return(api_to_csv(url))
    
api_to_csv("https://venmo.com/api/v5/public")

Running this function will create 100+ files on your computer but we still have to automate it!
For the automation we have used Apscheduler of Python to schedule tasks. Below is the sample scheduler code to print "hello world" after every 5 seconds.

from apscheduler.schedulers.blocking import BlockingScheduler
import datetime

def some_job():
    print("Hello World")
    print(datetime.datetime.now())

scheduler = BlockingScheduler()
scheduler.add_job(some_job, 'interval', seconds=5)
scheduler.start()

[
 {
  "name": "stdout",
  "output_type": "stream",
  "text": "Hello World\n2017-12-20 08:31:18.502273\nHello World\n2017-12-20 08:31:23.501694\nHello World\n2017-12-20 08:31:28.502143\nHello World\n2017-12-20 08:31:33.501585\nHello World\n2017-12-20 08:31:38.503171\nHello World\n2017-12-20 08:31:43.504585\nHello World\n2017-12-20 08:31:48.505167\nHello World\n2017-12-20 08:31:53.503578\nHello World\n2017-12-20 08:31:58.511812\nHello World\n2017-12-20 08:32:03.498899\nHello World\n2017-12-20 08:32:08.504150\nHello World\n2017-12-20 08:32:13.503669\nHello World\n2017-12-20 08:32:18.498747\nHello World\n2017-12-20 08:32:23.501515\nHello World\n2017-12-20 08:32:28.499996\nHello World\n2017-12-20 08:32:33.500166\nHello World\n2017-12-20 08:32:38.501800\nHello World\n2017-12-20 08:32:43.499564\nHello World\n2017-12-20 08:32:48.499563\nHello World\n2017-12-20 08:32:53.500164\nHello World\n2017-12-20 08:32:58.499722\nHello World\n2017-12-20 08:33:03.500837\nHello World\n2017-12-20 08:33:08.497938\nHello World\n2017-12-20 08:33:13.501267\nHello World\n2017-12-20 08:33:18.500235\nHello World\n2017-12-20 08:33:23.497763\nHello World\n2017-12-20 08:33:28.502114\nHello World\n2017-12-20 08:33:33.499198\nHello World\n2017-12-20 08:33:38.501426\nHello World\n2017-12-20 08:33:43.500664\nHello World\n2017-12-20 08:33:48.501903\nHello World\n2017-12-20 08:33:53.499416\nHello World\n2017-12-20 08:33:58.500171\nHello World\n2017-12-20 08:34:03.498846\nHello World\n2017-12-20 08:34:08.499330\nHello World\n2017-12-20 08:34:13.500103\nHello World\n2017-12-20 08:34:18.499377\nHello World\n2017-12-20 08:34:23.500401\nHello World\n2017-12-20 08:34:28.499612\nHello World\n2017-12-20 08:34:33.500976\nHello World\n2017-12-20 08:34:38.500106\nHello World\n2017-12-20 08:34:43.500404\nHello World\n2017-12-20 08:34:48.501349\nHello World\n2017-12-20 08:34:53.501847\nHello World\n2017-12-20 08:34:58.500325\nHello World\n2017-12-20 08:35:03.501061\nHello World\n2017-12-20 08:35:08.501592\nHello World\n2017-12-20 08:35:13.501427\nHello World\n2017-12-20 08:35:18.501180\nHello World\n2017-12-20 08:35:23.501314\nHello World\n2017-12-20 08:35:28.502168\nHello World\n2017-12-20 08:35:33.501680\nHello World\n2017-12-20 08:35:38.497236\nHello World\n2017-12-20 08:35:43.499432\nHello World\n2017-12-20 08:35:48.498568\nHello World\n2017-12-20 08:35:53.499473\nHello World\n2017-12-20 08:35:58.498436\nHello World\n2017-12-20 08:36:03.497465\nHello World\n2017-12-20 08:36:08.498967\nHello World\n2017-12-20 08:36:13.498772\nHello World\n2017-12-20 08:36:18.497115\nHello World\n2017-12-20 08:36:23.496763\nHello World\n2017-12-20 08:36:28.496967\nHello World\n2017-12-20 08:36:33.497429\nHello World\n2017-12-20 08:36:38.501424\nHello World\n2017-12-20 08:36:43.496841\nHello World\n2017-12-20 08:36:48.500259\nHello World\n2017-12-20 08:36:53.498417\nHello World\n2017-12-20 08:36:58.501419\nHello World\n2017-12-20 08:37:03.496691\nHello World\n2017-12-20 08:37:08.501313\nHello World\n2017-12-20 08:37:13.497264\nHello World\n2017-12-20 08:37:18.501239\nHello World\n2017-12-20 08:37:23.497984\nHello World\n2017-12-20 08:37:28.497683\nHello World\n2017-12-20 08:37:33.497062\nHello World\n2017-12-20 08:37:38.497463\nHello World\n2017-12-20 08:37:43.497020\nHello World\n2017-12-20 08:37:48.499097\nHello World\n2017-12-20 08:37:53.501761\nHello World\n2017-12-20 08:37:58.499752\nHello World\n2017-12-20 08:38:03.501579\nHello World\n2017-12-20 08:38:08.496237\nHello World\n2017-12-20 08:38:13.500931\nHello World\n2017-12-20 08:38:18.496430\nHello World\n2017-12-20 08:38:23.506581\nHello World\n2017-12-20 08:38:28.506565\nHello World\n2017-12-20 08:38:33.497006\nHello World\n2017-12-20 08:38:38.506572\nHello World\n2017-12-20 08:38:43.499588\nHello World\n2017-12-20 08:38:48.506444\n"
 },
]

Similarly we give our function api-to-csv() to Scheduler.it will create 100+ csv files on your desktop and prints their Created-at time as output as shown below

from apscheduler.schedulers.blocking import BlockingScheduler
import datetime
import requests
import csv
import json

# making a function for it
def api_to_csv(url):
    print(datetime.datetime.now())
    r = requests.get(url)
    data = r.json()
    public_data = data['data']

    lol = url.split("/")
    filename = lol[-1] + ".csv"
    
    with open(filename, 'w') as main_data:
        # create the csv writer object

        csvwriter = csv.writer(main_data)

        count = 0

        for temp in public_data:

              if count == 0:

                     header = temp.keys()

                     csvwriter.writerow(header)

                     count += 1

              csvwriter.writerow(temp.values())



        url = data["paging"]["next"]
        if url == "":
            pass


        return(api_to_csv(url))
    

    

scheduler = BlockingScheduler()
scheduler.add_job(api_to_csv("https://venmo.com/api/v5/public"), 'interval', hours=1)
scheduler.start()

[
 {
  "name": "stdout",
  "output_type": "stream",
  "text": "2017-12-20 20:32:33.958105\n2017-12-20 20:32:36.056567\n2017-12-20 20:32:37.696031\n2017-12-20 20:32:39.334850\n2017-12-20 20:32:40.971893\n2017-12-20 20:32:42.478251\n2017-12-20 20:32:44.220292\n2017-12-20 20:32:45.681311\n2017-12-20 20:32:47.141845\n2017-12-20 20:32:48.737739\n2017-12-20 20:32:50.237018\n2017-12-20 20:32:51.827506\n2017-12-20 20:32:53.670836\n2017-12-20 20:32:55.308800\n2017-12-20 20:32:56.714357\n2017-12-20 20:32:58.382515\n2017-12-20 20:33:00.016020\n2017-12-20 20:33:01.427892\n2017-12-20 20:33:02.814283\n2017-12-20 20:33:04.319055\n2017-12-20 20:33:05.750994\n2017-12-20 20:33:07.261144\n2017-12-20 20:33:09.026275\n2017-12-20 20:33:10.564273\n2017-12-20 20:33:12.085416\n2017-12-20 20:33:13.533011\n2017-12-20 20:33:15.174015\n2017-12-20 20:33:16.639188\n2017-12-20 20:33:18.028507\n2017-12-20 20:33:19.476915\n2017-12-20 20:33:21.108996\n2017-12-20 20:33:22.611758\n2017-12-20 20:33:24.012863\n2017-12-20 20:33:25.616063\n2017-12-20 20:33:27.672930\n2017-12-20 20:33:29.303410\n2017-12-20 20:33:30.870992\n2017-12-20 20:33:32.579040\n2017-12-20 20:33:34.012103\n2017-12-20 20:33:35.630611\n2017-12-20 20:33:39.955112\n2017-12-20 20:33:41.968799\n2017-12-20 20:33:43.556223\n2017-12-20 20:33:44.995266\n2017-12-20 20:33:46.479666\n2017-12-20 20:33:47.946240\n2017-12-20 20:33:50.400380\n2017-12-20 20:33:52.036958\n2017-12-20 20:33:53.587454\n2017-12-20 20:33:55.519904\n2017-12-20 20:33:56.970858\n2017-12-20 20:33:58.554474\n2017-12-20 20:34:00.163561\n2017-12-20 20:34:01.867900\n2017-12-20 20:34:03.336666\n2017-12-20 20:34:04.833714\n2017-12-20 20:34:06.370078\n2017-12-20 20:34:08.060204\n2017-12-20 20:34:11.290104\n2017-12-20 20:34:12.889743\n2017-12-20 20:34:16.736202\n2017-12-20 20:34:18.157015\n2017-12-20 20:34:19.689118\n2017-12-20 20:34:21.321612\n2017-12-20 20:34:22.875447\n2017-12-20 20:34:24.396238\n2017-12-20 20:34:26.236114\n2017-12-20 20:34:27.867929\n2017-12-20 20:34:29.926071\n2017-12-20 20:34:31.562701\n2017-12-20 20:34:33.204824\n2017-12-20 20:34:34.837859\n2017-12-20 20:34:36.379875\n2017-12-20 20:34:38.104345\n2017-12-20 20:34:39.496098\n2017-12-20 20:34:41.191615\n2017-12-20 20:34:42.825143\n2017-12-20 20:34:44.468346\n2017-12-20 20:34:47.508067\n2017-12-20 20:34:49.397037\n2017-12-20 20:34:51.022386\n2017-12-20 20:34:52.659271\n2017-12-20 20:34:54.294172\n2017-12-20 20:34:55.988239\n2017-12-20 20:34:57.982924\n2017-12-20 20:35:02.145702\n2017-12-20 20:35:03.718710\n"
 }
]

We have nice looking csv format as output after every 1 hour---cheers!...You can set it to days,hours,seconds just replace hours with seconds or days.....//

Screen Shot 1939-09-29 at 8.45.09 PM.png

Screen Shot 1939-09-29 at 8.45.13 PM.png

Python 2ApiJsonAutomationPython 3
Report

Enjoy this post? Give Siddhant Singh a like if it's helpful.

4
post comments3
Siddhant Singh
Python & JS Full Stack Developer (MERN & Django)
Experienced Python & JavaScript Developer with a demonstrated history of working in the field of web development. Skilled in Python, JavaScript, MERN, Django, Next.js, React, and data structure and algorithms. Strong engineering p...
Follow
Discover and read more posts from Siddhant Singh
get started
post commentsBe the first to share your opinion
MarkdownGitHub flavored markdown supported
submit
Poke
5 years ago

Hello.

It seems like this API does not work, as pagination in the API does not work. For example, going:

https://venmo.com/api/v5/public

and

https://venmo.com/api/v5/public?until=1513778837

return exactly the same data.

Any thoughts on this?

Reply
akshay kumar
6 years ago

Sir you have done a great job
M very thankful to you
U r such a helpful mentor

Reply
Siddhant Singh
6 years ago

Thanks Anytime!..//

Reply
Show more replies