Hieroglyph - the new coding language that can be done with a crayon

If a picture is worth a thousand words, then why do we code with words? Let's break down coding from my understanding as a sophomore at UC Berkeley majoring in CS and data science. Coding is largely a logical structure of words so that humans can understand it easily; binary is just a series of 1’s and 0’s representing a letter or command. For example this is “Hello world” in binary: “01001000 01100101 01101100 01101100 01101111 00100000 01110111 01101111 01110010 01101100 01100100”. Code is just shorthand for the binary command.

But since a word is worth who knows how many bits of binary ones and zeros, why can't we do that with photos or pictures? Why can't I draw up some stick figure design or create pseudo code and have it be its own coding language? This is my first blog post, so the structure is uncertain, but we'll be diving into this question as I try to create a hieroglyphic coding language.

The question that needs to be answered is: why haven't we made a picture-based coding language? The short answer: because we haven't been able to process photos and pictures that well and quikly until very recently.

Today, if you have an iPhone, you can simply use OCR on any photo that you have. You can select the text in your iPhone photo. So, let's say you take a picture of a stop sign; you can select that text. But even more complicated than that, by using nearest neighbor identifier algorithms and essentially measuring where the pixel is differentiated so much, you can tell an object in the foreground from the background. This can be seen if you have an iPhone: take a picture of yourself, something in the foreground, and you can pretty much make a sticker out of it. This is done by looking at where the borders of the pixels are and employing a complicated algorithm of nearest neighbor classification, among others.

But that hasn't been possible, especially in the early days of coding, nearest neighbor classification and image processing was more in theory then in practice. However, in today's world, picture-based coding languages are much more possible. My current idea is to build a language around a front-end development component like React. For those who don't know, React is a framework by Facebook that makes it easier to code web pages.The reason why it's so widely used is because it compacts everything into one place rather than scattered in CSS, and you can import a lot of libraries very easily.

Making the Hieroglyph compiler output a frontend development project start is an obvious choice, especially for something picture-based, as it's going to translate a picture to a website, with the intermediary steps being React, which is a framework. Alternatively, writing a program to convert a picture hierarchy code to Python code would be less intuitive, considering how to write complicated algorithms. It might be possible to make symbols for all these different classes, but you'd essentially need a decoder for it, or it would become really complex—the enemy of programmers. So, we'll stick with the front end, using this picture-based approach.

However, while the language will be built on top of React, it's going to be compiled by a Python program for ease of picture identification and decoding the picture, which is essentially the input for the back end. So, the order will be to stick drawing from Python to identify the photo then translate to words to be read by React. The benefit of this is that the code will finally look like it's in the first two stages—compiled and then final. Similar to how Java is written, once it passes the compile stage, it will never need to compile again, assuming it was written correctly. So, you don't have to convert it again from picture to Python.

To reiterate how this works, imagine using a physical wooden table—the table is the code. Building the table is the hard part. You need the skill, but once it's built, you never have to build it again. You can just put stuff on top of the table; you can just utilize the code. That's what this will be similar to java.

After an initial Google search, I found that a coding language like this doesn't exist. But I believe it could be very useful, especially in time crunch scenarios where a front-end developer needs just a little starting point, and then further customization of the front end could be done using conventional methods. Let's say you make a stick figure drawing of a website, and it comes out mostly right, but you want to change the color of the background or add some functionality with JavaScript that didn't come out perfectly as you hoped. That's where this approach could be beneficial.

Furthermore, teaching coding to kids through pictures would be an added bonus. As it exposes them to the mindset of coding without to much of the boring complicated syntax. As for the actual structure of the language, it would be best to start with the metadata. I'm thinking your literal legal signature could be shorthand for your coding name, similar to the readme. After that, perhaps a brief description in English of what the program is about and other information that's needed for simple logistics. It seems chronologically intuitive to start with the header or the navigation page at the top, as it's pretty much an industry standard to have one, especially when you want to navigate around the page without being stuck looking for links.

So, we're going to start with that, and maybe some shorthand drawing for symbols for loading animations or how you want the boxes to appear in color—that would also be a shorthand. From there, you would want the headers, essentially the main text of the page, like the introduction of the webpage or the first words that speak to the reader. It'll be written in English, but if you just want plain text, that's it. Animation shorthand, CSS shorthand drawings, and another field for the background would also be necessary.

As far as functionality components, input text, anything for your back end, you'd want shorthand for an input field, but have it be implied that this will be connected to a JSON, which will read some intermediary step between a back end written in, say, Python. Afterward, you'd want to see the result of the back end, which, depending on whatever the back end does, would be kind of tricky to implement. But for now, we can just say an output field—like some text that would be up for the front-end developer on a case-by-case basis. This language or this compiler will not cover this after-functionality.

An important part is divs and sections. I honestly think an easy way to do that would be like another piece of paper or another drawing. So, the first drawing would be the first section; the second drawing would be the second section. That seems intuitive. Bullet points would also be intuitive—just a simple bullet point would work. Oh, and finally, the footer of the webpage is pretty much an industry common standard, and this would probably have a lot of links. A shorthand basis for this could be directional boxes and maybe some other signatures for the links that you want to make. Or I wonder if you could do it all through JSON and say, "I'm expecting 10 links at the very bottom," so just make the space for it, and then in the JSON, you put, you know, 10 10 links and it looks parses through the jsonance is okay 10 links link one goes here spot one spot two and continue. Again image processing will just be done with python. I have to research libraries that I can use.

As far as project structure now this is going to work I feel like the obvious place to start is actually making the picture language and Define each symbol and Define all these identifiers for the image recognition program too recognize essentially like I'd be drawing Flags say hey look at this this is the signature for the page hey look at this this means a new section from there assuming if I could get the image recognition to work correctly I would just be needing to translate that into react to code which honestly would just be like a parser and with all these other fields that shouldn't be too hard and essentially what any code is even react in any coding terminal or like the S code anything a code editor is just a cracked notepad code is written could be written on notepad and then pass through coding language and would work just the same as the S code just vs code is has helpers that make it more intuitive catches your errors but at the end of the day if I wrote a python file in a notepad and everything was right it would still compile I'm in adults that is essentially just strings of text when I couldn't ages that's what goes on binary and we're back at the beginning.

Anyways send me an email if you're interested in collaboration. I'm very open to internships,let me know if you have any pointers. This will be my passion project in between classes which is going to be interesting for time management but I love it so I'm following my passion.

Work in progress with collaboration started at Cal Hacks 10.0, so far variable declaration and image recognition