Running behind the schedule since 1989.

I Created an Issue in a Flask-RESTful Repository, You Won't Believe What Happened Next

Two and a half years ago (Dec 4, 2015) I’ve written about a minor problem in Flask-RESTful, that puzzled and amused me. An issue was created on the same day. I then went on with my life, hoping for someone to step in and fix this trivial design inconsistency. Several people including the repo maintainer enthusiastically expressed support for proposed changes but no code was ever commited.

Fast forward almost a year (Oct 30, 2016), I noticed this issue while skimming through
my Github activities and decided to open a PR. I wasn’t using Flask at the moment, so I was doing it for the greater good. Having commited some code and hoping for the maintainer to review it ASAP, I switched to other commitments. Needles to say there was no review in several days, weeks and even months. Feeling more and more irritated with the whole affair, I emailed the maintaner and several other people, whose PRs have been merged recently - to no avail. I gave up and decided to finally let it go.

Fast forward almost two years (Jun 6, 2018) the maintainer of the repository suddenly started reviewing the code. It is now merged for good and I’m glad to have contributed to the project, but somehow this whole ordeal of trying to commit my trivial change for almost 2.5 years makes me worried about the ever-increasing entropy of Open Source. Perhaps several thousand new libs and frameworks were created last couple of minutes. Do we have the guts to support and maintain it all? Could it be that we got carried away with a sexy idea of starting our own new thing and developed intolerance for the actual grind and commitment?

Limitations of the Beeminder Model

I’ve been having somewhat on and off relationship with Beeminder. I’m not going to waste time explaining what it is as if it wasn’t done again and again all over the internet, but for the sake of keeping you on this page: it is a service that keeps you motivated, using commitment contracts and a bit of money pledging. On one side I’m convinced this is absurd model and we are rational enough to achieve our goals without pledging money on the progress. On the other side though: I can’t stop procrastinating on my rationally chosen goals and somehow this relatively simple behavioral trick does wonders. I can tell the difference in productivity between the time I’m actively using Beeminder and the time I’m off, and this difference is as problematic as this is great: there is no long term effect to it all. It goes back to square one the moment you decide to archive your goal or stop using the service altogether. Before I go into detail, a little disclosure is due: I’m no psychologist or economist and my understanding is based on pretty anecdotal evidence. I’m not going to pretend this all is somewhat objective and smartass, at least I don’t want you to have that impression.

So let’s start and create a goal. Actually, let’s create two goals: a Do More (steps done) and a Do Less (cups of coffee). We’re tracking them for some time and we walk a lot and are extremely decaffeinated. After a couple of months we decide we don’t need Beeminder anymore. We’ve grown over it. However, the next couple of days we suddenly notice that we just don’t do as much walking as before and drink a lot of coffee, probably even more than we used too. What happened? In my understanding Beeminder goal is a little economy in itself, an equivalent of Monopoly money. You have a commodity (cups of coffee, day of sedentary lifestyle, etc) that you attempt to limit artificially through placing a price on that. Thus the economy is born. Especially if you never retroratchet your goals it may get a little silly, as you start accumulating your buffer and making spending decisions. For example: one cup of coffee costs 1 step of Beeminder graph, you can accumulate steps available to you by saving, you can also inflate and deflate your Beeminder step, or you can get some buffer in exchange for money. You always have the price for your negligence, so that the negligence becomes a commodity in itself.

When you occasionally deflate the step too much by removing the goal or making it too easy — you create a situation where the consumer (you) is subconsciously driven towards increase in consumption. You’re creating an economical situation that has psychological consequences. In my opinion the problem is that eventually you remove your authentic self from the loop: “I don’t drink this much coffee because it’s unhealthy” becomes “I don’t drink coffee because I don’t have enough magic or real coins for that”. This substitution may be effective at first, but in the long run it eventually comes to catching up on coffee when the artificial limitation drops. When you suddenly have all the money in the world can you find enough willpower not to buy as much as you can carry? I noticed this effect with various goals: not only it all returned to the original level, it got worse for some time. You may have noticed similar problem with diets, as soon as you get off, all the weight is back and sometimes it gets even worse. Beeminder treats the symptoms without fixing the underlying problem and that is the greatest limitation of this model.

What’s most troubling about all that, I don’t really have a solution in mind, only a hint of it. In his book Existential Psychotherapy Irvin Yalom suggests a near death experience as a method of becoming a more motivated and authentic self. Of course it’s a bit extreme, although there are exercises to emulate this effect to some degree. Somehow this idea also  correlates with Tim Urban’s  brilliant Ted talk on procrastination which makes me think there is something about this idea. In my opinion if there is really somewhat universal key to finding your real authentic goals and sticking to them in the long run, it is  somewhere in the area of realizing our finitness and creating a short feedback circuit to review every day in the perspective of the time we still have. Making every day the first day of the rest of your life. But there is still no clear methodology, that would help you avoid being lulled into the mundane, the comfort zone of seemingly endless life, full of mindless routine and the ensuing procrastination. Until we are there, let’s Beemind our way through.

Aspiring Photographers are Quitting Instagram en Masse and Here's Why

Full disclaimer: the title is a total clickbait. Still I’m very concerned about the current state of Instagram and the direction the site has taken. Here are some of my grievances:

  • I haven't seen anyone but bots in a while. Sure, that's a gross exaggeration, but it certainly feels like actual people are in the minority on the site. Most accounts that like or follow me are spambots or some low-tier SMM guys from developing world.
  • My major grievance is lack of posting in official API. Sure, there are unofficial private API wrappers, but why should users jump through the hoops? Limiting access to CRUD operations in public API is walled garden approach at its finest.
  • No links. Instagram prohibits adding clickable links and copying in both descriptions and comments. I understand they're doing it to minimize spam, but it's like getting rid of intestines to avoid diarrhea. Also, see the first point. How much should it take before ineffectiveness of this approach is evident? In the meanwhile there is no way for the regular user to share a link to a bigger story.
  • Instagram still reduces the quality of pictures and ruins your videos. Sure, it's much better now than it used to be, but it's still far from ideal. Plus, what's up with quality of Instagram pics on Facebook? Is this by design? Because I have no other explanation how it can stay unfixed for so long.
  • It doesn't cater towards its original target audience - the amateur photographers. Instagram is no longer a network for good phone photography (or attempts at that), most pics one sees today are selfies, which says a lot about the current target audience of the service. Introduction of Stories (which is 100% Snapchat rip-off) cemented this trend.

I’m still using the service for sharing what I deem interesting shots and occasional videos, but I’m also using Flickr as some kind of mirror feed and to be honest Instagram starts to feel like dead weight.

TIL: Mongo Sucks

MongoDB $nin and $ne and $exists: false queries are super expensive. I mean there is a point at which they are basically unusable. And I learned it the hard way, having developed a service, that relies heavily on this kind of queries to emulate queue-like workflow. 5+ million documents later and I’m hurriedly adding indexes for all flags that take more than 60 seconds to query for.

Information on this issue is limited to a short FAQ entry at Mongo docs and a couple StackOverflow answers. That’s it. Come on, it should be the first thing people see, when they open Mongo docs - a huge dialog you should scroll through three times before you can click accept.

Overriding Default Werkzeug Exceptions in Flask

Update 15.06.2018: The issue described in this post is now a history, read the epic story of my struggle to get it fixed.

Let’s play a game here. What HTTP code is this exception:

  "message": "The browser (or proxy) sent a request that this server could not understand." 

No no, you don’t look at the code in response! That’s cheating! This is actually a default Werkzeug description for 400 code. No shit. I thought something is bad with my headers or encryption, but I would never guess simple Bad request from this message. You could use a custom exception of course, the problem is, that the very useful abort(400) object (it’s an Aborter in disguise) would stick with the default exception anyway. Let’s fix it, shall we?

There may be several possible ways of fixing that, but what I’m gonna do is just update abort.mapping. Create a separate module for your custom HTTP exceptions custom_http_exceptions.py and put a couple of overridden exceptions there (don’t forget to import abort we’ll be needing that in a moment):

from flask import abort
from werkzeug.exceptions import HTTPException

class BadRequest(HTTPException): 
    code = 400
    description = 'Bad request.'

class NotFound(HTTPException): 
    code = 404
    description = 'Resource not found.' 

These are perfectly functional, but we still need to add it to the default abort mapping:

    400: BadRequest, 
    404: NotFound 

Note that I import abort object from flask, not flask-restful, only the former is an Aborter object with mapping and other bells and whistles, the latter is just a function.

Now just import this module with * to your app Flask module (where you declare and run your Flask app) or someplace it would have a similar effect on runtime.

Note that you also should have the following line in your config because of this issue:

ERROR_404_HELP = False 

I’m not sure why this awkward and undocumented constant isn’t False by default. I opened an issue on GitHub, but no one seems to care.