Firing a million async requests with Python

There is no reason you'd ever need to fire a million requests to a single server so don't do this. like totally. don't. do. this.

import asyncio
import aiohttp

async def get_stuph(url, session, i):
    print(f"Firing request #{i}. brrr.")
    await session.get(url)

async def main():
    url = "http://some_url.nope"
    async with aiohttp.ClientSession() as session:
        _ = await asyncio.gather(
            *[
                get_stuph(url, session, i)
                for i in range(1_000_000)
            ]
        )

asyncio.run(main())


Visual semantic search for a million NFTs with Alchemy, OpenAI's CLIP & Pinecone — easy as A⁠-⁠P⁠-⁠E!

A gif showing bounding box drawn over the image of a cougar in a dark forest
NFTopia

It was Spring '22. The snow was meltin', the birds were singin', and my fellow ape Xi Chen was deep in the rabbit hole of crypto & NFTs. As he navigated this labyrinth, he often found himself screaming "ooooo-oo-ah-ah-oooo" which is ape-speak for — "Yo, why can't I simply search for NFTs by describing what's in the image". Why? I can only speculate but I presume he wanted to search for something like "ape driving a lambo". I mean, I know that's what I'd do! As a bonus, if there were none, when we did get our first Lambo trading NFTs, we could sell a picture of us driving it as an NFT to get a second Lambo! An ape can dream!



Check file size before downloading it with Python

If you're yolo-ing on the web and downloading a lot of content, especially arbitrary media files using a crawler, it might be useful to first check the mimetype & filesize before downloading it.

To do this with Python's requests module, you'll have to set stream=True and examine the headers for size & mime type. Following that, you can retrieve the content.


Set a retry strategy for Python requests

If you're overly enthuiastic with your requests to a server, it can get passive aggressive and give you the silent treatment or get overwhelmed and ask for a vacation. To mitigate that, give it some space with a retry strategy. Here's one such strategy with Python's requests package.


Building a simple AI-powered, human-in-the-loop system to manage wildlife camera trap images & annotations

A gif showing bounding box drawn over the image of a cougar in a dark forest Cougar in Purisima Creek Redwoods Preserve by Felidae Conservation Fund

In the summer of 2021, after three years of volunteering with Code Nation as an instructor and a brief stint with Code for SF, I was ready for something new! Wading through the web on a mission to find a new nonprofit, I stumbled onto a LinkedIn post from Felidae Conservation Fund. As their name suggested, they were a wildlife research & conservation organization that studied wild cats, specifically mountain lions. For someone who grew up on David Attenborough, the word "conservation" alone was enough to spark a sense of excitement but when I learned that they needed a cloud-based AI solution to improve their data pipelines, I was sold!

In this post, I briefly chronicle my journey designing and building this AI-powered, human-in-the-loop system to manage Felidae's camera trap images & annotations. If you're interested in the role of technology in wildlife conservation, building computer vision applications or working with nonprofits, you might find this post useful. If you want a high-level overview before jumping into this post, you can also check out this slide deck.