Kicking the Bucket

Recognizing and overcoming harmful classifications

A challenging thing about being human is, (multinomial; binary or similarly discrete-classed) logistic classification is too often employed where a continuous classifier would better serve us. There are times when logistic regression is the right tool -- when there is an absolute answer (e.g. is this shape a square or a triangle). However, in many real-world scenarios, we compress the dimentionality of spectrums (autism, skin color, gender, personality-types) into rigid, over/under-fitting generalizations whose classifications serve to satisfy our political goals (a generalization we make at sometimes great cost of real people).

In some cases, generalization is a consequence of real world limitations. In fact, this is a great example of where algorithm complexity analysis (rigorously quantifying the properties of algorithms) can give us a foundation and better understanding of real-world optimization. In practice, should a student fall just short of an "autistic" classification, they may be entirely ineligible (e.g. IEP) to receive or benefit from special instruction. That is to say, the model is limited in that not everyone benefits to the degree proportionate to their needs. An analogous continuous, linear approach, would be a custom curriculum for every student which addresses their every individuality. As you can imagine, while this has tremendous benefits (and we are + will see technology increasingly achieve this) it is challenging to standardize and costly to implement. It's like making dinner for a family. The optimal case is to make everyone what they want, but preparing several different dishes with different ingredients takes a lot of time, money, and skill. As a result, an attempt is made to serve the majority by choosing a single menu (outcome). Restaurants are a model which serves to capitalize this inefficiency.

Corollary to modelling something continuous via generalized logits, there will (by definition, otherwise it wouldn't be continuous) exist inefficiencies (mis-labellings) and gaps wherein thrive the Hacker. You see it with tax brackets; someone creatively structuring their finances to pay less tax. Or in gaming a job interview; preparing for the expected/common questions. Even the way some people dress or speak are attempts at triggering or activating some sigmoid (logistic) function to appear / be perceived as associated with a certain class or set of values.

Society tends to classify Hackers, posers (I'll call them social engineers), and the likes, as evil, as they (presumably) opportunistically exploit our societal expectations/contracts/promises (mutual agreements/understandings) for personal gain. Some people simply like to better understand our contracts (learning for the sake of learning), and what these inefficiencies are or entail. Some people do indeed want to discover ways of drawing or deriving value. One could argue, this inquisitive hacker spirit (irrespective of the outcome) is both the biggest threat and the most essential imperative to our advancement as a civilization. As inefficiencies are discovered, we the people (more broadly, 'the government') devise new legislature, social contracts, and or popularize trends (read: movements) to either minimize or ubiquitize this exploitation for optimal benefit of society as a whole (e.g. drug enforcement, police).

Of course, Hacker itself is a binary logistic, perhaps we can do a little better and make it ternary (admittedly still not perfect). The difference between white hat, grey hat, and black hat hackers, in my mind, is that the white hats discover inefficiencies between classes or how contracts can physically be subverted or ignored, but don't themselves act towards exploiting them. The grey hats live in the ambiguity or inefficiencies of social contracts (blurring the lines) and attempt to legally derive benefit. The black hats ignore social contracts altogether in pursuit of gains.

While hackers are indispensable for discovering and solving inefficiencies, capitalism is one of the driving factors for sustaining such solutions at scale. It's no mystery why many hackers are entrepreneurs, and vis versa. The general public (99%) may feel discouraged by capitalism because in "ameliorating" (reallocating) the system as a whole, their disproportionate gains may be compromised/diminished. It's unfortunate that re-drawing the lines can results in negative side effects, there are likely interesting strategies to address this (e.g. exchanges + recompense, like eminent domain), and this should be addressed in essays by people much more qualified than I. On the other hand, slavery was one of those lines. Women voting was one of those lines. And some lines need to be redrawn and others removed altogether. Another case against capitalism is, some ventures are, in ways other than finance, a net negative on the world. Economics is a very difficult field and I prefer addressing problems I can rationalize against specific algorithms, so again I'll leave this legitimate concern to someone more qualified.

Responses

Kartik Agaram offers the following in response: White-hat version | Grey/black-hat version

Incidentally, I'd like to quibble about your definitions of the hats. I'll suggest these instead: a) White hat: motivated by self-interest, constrained by own reputation with others. b) Grey hat: motivated by self-interest, constrained by what's legal. c) Black hat: motivated by self-interest, constrained by imagination.

Followup

On Wednesday, February 18, 2015 I wrote an essay[1] on: "Recognizing
and overcoming harmful classifications"[2].

The main point of the essay is, we (as people) tend to use heuristics
to compress highly dimensional problems (e.g. voting for a politician)
into a single, logistic, binary Yes/No answer (e.g. democrat,
republican). It's a type of stereotyping which aims to keep us "safe",
many times at the expense of overfitting (hurting those people or
ideas which do not fit stereotypes).

How many ideas and views can one possibly group under the single word
"democrat" or "republican"? If I said, "the right to choose" which
would I be speaking of? How is it any different from having a single
word "Christian", "Muslim", or "Jewish"?

My claim is, these types of over-simplifying, one-size-fits-all
classifications are damaging to us. They forces us to make distinction
about entire classes of people which (by the sheer number of
dimensions involved) cannot be comprehensive and statistically
accurate. And what's more, it prevents us from identifying the actual
underlying qualities which are in fact problematic. As importantly, it
forces us to unecessarily compromise our ideals as it intertwines and
convolutes the fates of separate ideas and issues into a single
decision. Suddenly voting democrat means passing marijuana reform and
banning guns, even if you only support or oppose one of those issues.

Technology is already so advanced, there are demonstratably better
ways for us to represent and express our ideas, as a point along a
spectrum (or multiple spectrums -- see a radar graph[3]), rather than
an over-fitting yes/no dichotomy.

At this point, I genuinely believe we're (Americans, me included)
being lazy. And we are justifying our laziness through freedom of
speech.

I think there are many reasons so many Americans don't vote. If you
would like all the reasons enumerated, and a proposed solution,
consider reading my essay: Towards More Efficient Political
Voting[3.5]) First, it's an ordeal. Many people have trouble enough
encouraging themselves to cook meals and do laundry. We're also
afraid. We're afraid of voting in a way where we'll be wrong. How can
we possibly be right, selecting a single person and their cabinet to
be successful in solving over 100[4] social issues? This can be
mitigated by giving people greater granularity to weigh in on issues
they care about. Jan Paul Posma and Ben Woosley are helping the world
in this regard with their contributions to Brigade.

We need to make technologies which both lower the frictional barriers
to making "stereotype"-like decisions, and at the same, time impose
regulation and process which (doesn't limit freedom of speech, but)
requires people -- through process itself -- to not make lazy, or
blatantly uninformed stereotypical decisions.

And we're not just talking about voting and politics. I'm talking
about how we treat our fellow human beings. I'm talking about the
dangers of subscribing to a blanket political party, religion, or any
other group of hundreds of thousands of people, each who think
differently, and different degrees of things, yet unite under a single
label, a single color, and a single belief -- even if the underline
doctrines of that group are inconsistent.

Naomi Most posted an interesting youtube video of a group of German
interviews who disguised the Bible as the Quran[5] which helped me
realize something about myself (and of others). I realized that we
tend to compress the dimensionality, and over-simplify the complexity,
of all sorts of things (including books) into single
classifiers. Books are perhaps the best example. The entire point of a
book is to (hopefully) thoughtfully express what cannot be said in a
single word or sentence.

It's not our fault. There's far too much knowledge for us to possibly
consume or to reason with. But it is our fault. For some reason, even
though we acknowledge there's too much information to possibly
consider, we want to be the authority. We want to believe the
arbitrary classifications to which we subscribe to are well-fitting
and correct. So much so, that we lie to ourselves (take liberties with
the underlying points) in order to make the high-level points
consistent again.

It's the reason we're so willing to lie about knowing a band we don't
know (because they don't even exist)[6]. It's the reason we're willing
to believe one of the same coffee is better because it has a higher
price point[7].

Yes, people have flaws. And that's OK. But the point isn't to
highlight and dwell on our flaws. It's to do regression testing. It's
to augment our capabilities and to make us better. To learn the ways
in which we are flawed, and to create constructs which encourage us to
not be lazy, to help us make better decisions. Which encourage us to
take out our phones and spend 5 minutes checking the accuracy of
something before making a decision (or responsibly not saying anything
at all).

We need tools for navigating paths through knowledge, and we need to
not be afraid to use them. We need people to build tools which help
people make more informed decisions. And we need a society which
encourages people to be accountable to reality and to each other, and
not the arbitrary words we use to separate each other.

[1] Kicking the Bucket https://michaelkarpeles.com/ess…/politics/kicking-the-bucket
[2] Original post: https://www.facebook.com/michael.karpeles/posts/10102139730729330 [3]
Radar Graph example
http://jpgraph.net/…/manuals/c…/images/fixscale_radarex1.png [3.5]
Towards More Efficient Political Voting
https://michaelkarpeles.com/…/towards-efficient-political-v…
[4] US Social Issues
http://catdir.loc.gov/catdir/toc/ecip0516/2005018778.html
[5] Bible disguised as Quran
https://www.youtube.com/watch?v=zEnWw_lH4tQ
[6] Lie Witness News - Coachella 2013
https://www.youtube.com/watch?v=W_IzYUJANfk
[7] New $7 Cup of Coffee at Starbucks
https://www.youtube.com/watch?v=HxlGI4OzeBk