Kicking the Bucket
Recognizing and overcoming harmful classifications
A challenging thing about being human is, (multinomial; binary or similarly discrete-classed) logistic classification is too often employed where a continuous classifier would better serve us. There are times when logistic regression is the right tool -- when there is an absolute answer (e.g. is this shape a square or a triangle). However, in many real-world scenarios, we compress the dimentionality of spectrums (autism, skin color, gender, personality-types) into rigid, over/under-fitting generalizations whose classifications serve to satisfy our political goals (a generalization we make at sometimes great cost of real people).
In some cases, generalization is a consequence of real world limitations. In fact, this is a great example of where algorithm complexity analysis (rigorously quantifying the properties of algorithms) can give us a foundation and better understanding of real-world optimization. In practice, should a student fall just short of an "autistic" classification, they may be entirely ineligible (e.g. IEP) to receive or benefit from special instruction. That is to say, the model is limited in that not everyone benefits to the degree proportionate to their needs. An analogous continuous, linear approach, would be a custom curriculum for every student which addresses their every individuality. As you can imagine, while this has tremendous benefits (and we are + will see technology increasingly achieve this) it is challenging to standardize and costly to implement. It's like making dinner for a family. The optimal case is to make everyone what they want, but preparing several different dishes with different ingredients takes a lot of time, money, and skill. As a result, an attempt is made to serve the majority by choosing a single menu (outcome). Restaurants are a model which serves to capitalize this inefficiency.
Corollary to modelling something continuous via generalized logits, there will (by definition, otherwise it wouldn't be continuous) exist inefficiencies (mis-labellings) and gaps wherein thrive the Hacker. You see it with tax brackets; someone creatively structuring their finances to pay less tax. Or in gaming a job interview; preparing for the expected/common questions. Even the way some people dress or speak are attempts at triggering or activating some sigmoid (logistic) function to appear / be perceived as associated with a certain class or set of values.
Society tends to classify Hackers, posers (I'll call them social engineers), and the likes, as evil, as they (presumably) opportunistically exploit our societal expectations/contracts/promises (mutual agreements/understandings) for personal gain. Some people simply like to better understand our contracts (learning for the sake of learning), and what these inefficiencies are or entail. Some people do indeed want to discover ways of drawing or deriving value. One could argue, this inquisitive hacker spirit (irrespective of the outcome) is both the biggest threat and the most essential imperative to our advancement as a civilization. As inefficiencies are discovered, we the people (more broadly, 'the government') devise new legislature, social contracts, and or popularize trends (read: movements) to either minimize or ubiquitize this exploitation for optimal benefit of society as a whole (e.g. drug enforcement, police).
Of course, Hacker itself is a binary logistic, perhaps we can do a little better and make it ternary (admittedly still not perfect). The difference between white hat, grey hat, and black hat hackers, in my mind, is that the white hats discover inefficiencies between classes or how contracts can physically be subverted or ignored, but don't themselves act towards exploiting them. The grey hats live in the ambiguity or inefficiencies of social contracts (blurring the lines) and attempt to legally derive benefit. The black hats ignore social contracts altogether in pursuit of gains.
While hackers are indispensable for discovering and solving inefficiencies, capitalism is one of the driving factors for sustaining such solutions at scale. It's no mystery why many hackers are entrepreneurs, and vis versa. The general public (99%) may feel discouraged by capitalism because in "ameliorating" (reallocating) the system as a whole, their disproportionate gains may be compromised/diminished. It's unfortunate that re-drawing the lines can results in negative side effects, there are likely interesting strategies to address this (e.g. exchanges + recompense, like eminent domain), and this should be addressed in essays by people much more qualified than I. On the other hand, slavery was one of those lines. Women voting was one of those lines. And some lines need to be redrawn and others removed altogether. Another case against capitalism is, some ventures are, in ways other than finance, a net negative on the world. Economics is a very difficult field and I prefer addressing problems I can rationalize against specific algorithms, so again I'll leave this legitimate concern to someone more qualified.
Incidentally, I'd like to quibble about your definitions of the hats. I'll suggest these instead: a) White hat: motivated by self-interest, constrained by own reputation with others. b) Grey hat: motivated by self-interest, constrained by what's legal. c) Black hat: motivated by self-interest, constrained by imagination.
On Wednesday, February 18, 2015 I wrote an essay on: "Recognizing and overcoming harmful classifications". The main point of the essay is, we (as people) tend to use heuristics to compress highly dimensional problems (e.g. voting for a politician) into a single, logistic, binary Yes/No answer (e.g. democrat, republican). It's a type of stereotyping which aims to keep us "safe", many times at the expense of overfitting (hurting those people or ideas which do not fit stereotypes). How many ideas and views can one possibly group under the single word "democrat" or "republican"? If I said, "the right to choose" which would I be speaking of? How is it any different from having a single word "Christian", "Muslim", or "Jewish"? My claim is, these types of over-simplifying, one-size-fits-all classifications are damaging to us. They forces us to make distinction about entire classes of people which (by the sheer number of dimensions involved) cannot be comprehensive and statistically accurate. And what's more, it prevents us from identifying the actual underlying qualities which are in fact problematic. As importantly, it forces us to unecessarily compromise our ideals as it intertwines and convolutes the fates of separate ideas and issues into a single decision. Suddenly voting democrat means passing marijuana reform and banning guns, even if you only support or oppose one of those issues. Technology is already so advanced, there are demonstratably better ways for us to represent and express our ideas, as a point along a spectrum (or multiple spectrums -- see a radar graph), rather than an over-fitting yes/no dichotomy. At this point, I genuinely believe we're (Americans, me included) being lazy. And we are justifying our laziness through freedom of speech. I think there are many reasons so many Americans don't vote. If you would like all the reasons enumerated, and a proposed solution, consider reading my essay: Towards More Efficient Political Voting[3.5]) First, it's an ordeal. Many people have trouble enough encouraging themselves to cook meals and do laundry. We're also afraid. We're afraid of voting in a way where we'll be wrong. How can we possibly be right, selecting a single person and their cabinet to be successful in solving over 100 social issues? This can be mitigated by giving people greater granularity to weigh in on issues they care about. Jan Paul Posma and Ben Woosley are helping the world in this regard with their contributions to Brigade. We need to make technologies which both lower the frictional barriers to making "stereotype"-like decisions, and at the same, time impose regulation and process which (doesn't limit freedom of speech, but) requires people -- through process itself -- to not make lazy, or blatantly uninformed stereotypical decisions. And we're not just talking about voting and politics. I'm talking about how we treat our fellow human beings. I'm talking about the dangers of subscribing to a blanket political party, religion, or any other group of hundreds of thousands of people, each who think differently, and different degrees of things, yet unite under a single label, a single color, and a single belief -- even if the underline doctrines of that group are inconsistent. Naomi Most posted an interesting youtube video of a group of German interviews who disguised the Bible as the Quran which helped me realize something about myself (and of others). I realized that we tend to compress the dimensionality, and over-simplify the complexity, of all sorts of things (including books) into single classifiers. Books are perhaps the best example. The entire point of a book is to (hopefully) thoughtfully express what cannot be said in a single word or sentence. It's not our fault. There's far too much knowledge for us to possibly consume or to reason with. But it is our fault. For some reason, even though we acknowledge there's too much information to possibly consider, we want to be the authority. We want to believe the arbitrary classifications to which we subscribe to are well-fitting and correct. So much so, that we lie to ourselves (take liberties with the underlying points) in order to make the high-level points consistent again. It's the reason we're so willing to lie about knowing a band we don't know (because they don't even exist). It's the reason we're willing to believe one of the same coffee is better because it has a higher price point. Yes, people have flaws. And that's OK. But the point isn't to highlight and dwell on our flaws. It's to do regression testing. It's to augment our capabilities and to make us better. To learn the ways in which we are flawed, and to create constructs which encourage us to not be lazy, to help us make better decisions. Which encourage us to take out our phones and spend 5 minutes checking the accuracy of something before making a decision (or responsibly not saying anything at all). We need tools for navigating paths through knowledge, and we need to not be afraid to use them. We need people to build tools which help people make more informed decisions. And we need a society which encourages people to be accountable to reality and to each other, and not the arbitrary words we use to separate each other.  Kicking the Bucket https://michaelkarpeles.com/ess…/politics/kicking-the-bucket  Original post: https://www.facebook.com/michael.karpeles/posts/10102139730729330  Radar Graph example http://jpgraph.net/…/manuals/c…/images/fixscale_radarex1.png [3.5] Towards More Efficient Political Voting https://michaelkarpeles.com/…/towards-efficient-political-v…  US Social Issues http://catdir.loc.gov/catdir/toc/ecip0516/2005018778.html  Bible disguised as Quran https://www.youtube.com/watch?v=zEnWw_lH4tQ  Lie Witness News - Coachella 2013 https://www.youtube.com/watch?v=W_IzYUJANfk  New $7 Cup of Coffee at Starbucks https://www.youtube.com/watch?v=HxlGI4OzeBk