23 Things I Learned While Making My Own Website From Scratch
Ok, so here are some things I learned while building my website from scratch. And yeah, when I say from scratch, I mean I coded up my front end, back end, and database logic by hand with minimal frameworks. Note I'm not saying 'no frameworks' because that wasn't the point of this project. The main point was to have a 'property' in cyberspace that I have complete control over. And yeah, since it's my property, performance is one of the main focuses. With that said, I wasn't about to write the whole server in C or something like that. This wasn't that type of project. The focus is on The Art.
So the stack I decided on was Go for the back end and HTML templating, HTMX for stuff that needed to be loaded without a page refresh, vanilla JS where UI functionality is needed, Postgres for the database, Docker to package it all up, and Nginx to handle the server routing. And it all lives on a single VPS. As a matter of fact, one of the cool things about this setup is that I can spin up multiple different sites on the same VPS just like this one. You might be thinking Docker is over-kill but nah, it's not. Like I said, the whole point is having a performant digital property. So I wanted something that, not only wasn't tied to any specific hosting platform (like Heroku, Vercel, Netlify, whatever), but that I could basically deploy to bare metal in one command. Didn't wanna have to worrry about installing a database or tools I use like ffmpeg or anything like that. Docker is pretty much THE answer for that. If, for some reason, my hosting provider ever went down, I could have my site up and running on another one in minutes. So yeah, lemme break down some of the stuff I learned while doing this.
1. Static HTML is GOOD
Before you even start thinking about how you're gonna make a back-end, ask yourself, 'Do I even really need one?'. Personally, I felt like I did because ultimately I wanted my site to mainly be a place for me to distribute my music and art. So that meant I wanted it to also function as a tool for me to format my audio files from WAV format (which is completely uncompressed) to MP3, by just uploading them to my site. But the reality is, I could also just export both versions from my DAW. So, did I really NEED that functionality... Nah. But it is really nice to have. You just need to evaluate if it's worth it to you, because I won't lie. It's kinda involved to create a dashboard where you manage all your stuff like this. But here's how mine works if you're interested.
I have FFMPEG installed on my server, so whenever files are uploaded, they're automatically processed based on what file type they are. Images and videos get shrunk (so that I have a smaller version to show on the page and save bandwidth/page loading time). Videos also get a thumbnail image extracted from them that I can use on my site pages instead of initially loading the whole video. WAV files get MP3 verions created. Right now, I only create 1 smaller size for images/thumbnails (max 500px wide) and for videos and MP3s, 1 compressed version (192kbs per second), but I can easily update this in the future so that more sizes are created. I could also add some controls in the UI that allow specifying the additional sizes and formats created. I'm also retaining the orignal files so I have them for displaying the high-res version if I need to. I'm not really displaying the original resolution images or videos right now, just the WAVs of the music, but I have them on the server so I could easily do that in the future if I decide.
2. I'm glad I didn't use a framework, because they come with so much bloat (cognitively too)
So, just say you're like me and you DO decide that you wanna build a back end, complete with a database and all that. The industry standard these days is to use some type of 'Web Framework' to build your project. So you do a bit of Googling and see that stuff like React, Vue, or Svelt are pretty popular. But then you get into development with those, and you realize, they're only for the 'front end' of the web app/site. So then you realize, ok, now you need to build a 'back end' that's a REST api that's responsible for sending content back and forth to the front end. So you now, you need to use something like Node.js with Express. Then, after all that, you realize your site doesn't really handle web crawlers like Google well because the pages have no content when they're initially accessed since all content is requested through asynchronous HTTP calls to the api. You also don't really have separate pages for your site, which can make sharing content on Social Media kinda tricky because you'll need to rely on a ton of url query parameters to determine what actually shows up on your 'Single Page Application' when it's shared. But even after you do that, you realize you still can't really share pages properly on Social Media because the html tags they read to determine how your 'share card' looks are read from the initial HTML request. Now you finally realize, you need something called SSR, or 'Server Side Rendering' to fix all this. So, you can implement that yourself (I've done it before), but be warned, it is kinda hacky, and you'll end up writing a lot of duplicate code. So finally! (for real this time), you realize what you actually need is something like NextJS for React, or Nuxt for Vue. And after all that, you still have to maintain two separate applications if you went with the whole 'separate api' back end model. (trust me, this model only really makes sense if you're making a web app that's also gonna be a native mobile app. It's kinda pointless for websites).
Or, maybe you decided to go another route. You saw something about 'complete' frameworks that would contain everything you need for the front end, the back end, the database, and also even with the ability to do some API stuff too. That would be stuff like Django (Python), Laravel (PHP), or maybe even Wordpress(PHP). Now, these aren't bad per se but they have some drawbacks that you should be aware of before getting too invested. They actually do come 'out of the box' with better suppport for building a full working site with back end and front end, however. From what I recall, their primary function was to be 'Rapid Development' frameworks. That basically means they're awesome for getting prototypes of web apps up and running quickly. The downside is that they can be lacking in performance due to coming bundled with a bunch of code that your project probably doesn't need. The are designed to handle so many generic use cases. Now, this isn't a terrible thing but I gotta be honest. This was more useful before the age of AI. I'm not advocating for slopping something together, but for building 'rapid prototypes' that will need to be refined later... it is kinda good at that. Just know that if you choose one of these, you might wanna go in do a lot of stripping out of unnecessary code later on.
3. The back end language and database don't really matter all that much
Unlike frameworks, you don't really have a choice for the front end languages at all, it's still strictly HTML and JavaScript/TypeScript. I don't wanna hear anyone 'well actually' me about WASM either. I'm not writing a f-ing an form in C. For back end though, there's a metric butt-load of choices. Anyway, for my back end, I chose Go (Golang) partially because I have just been wanting to learn it, but also because it's faster than stuff like PHP, Python, and JavaScript. It also enforces variable types without needing a pre-processor like JavaScript does with Typescript. PHP and Python support them now but ehhh, it's kinda iffy. Still, they don't have pointers and I wanted a language that exposed those too. But honestly, if you do decide to build a back end, it doesn't really matter that much what you choose. The speed/performance difference isn't all that noticicable to the user because 9 times out of 10, the actual bottleneck will be their network connection, not really your code execution speed. But yeah, if you wanna learn something in particular, building a site back end is an awesome way to do it.
4. Just use Static HTML wherever it makes sense to!
Ok, back to this! Even if you decide to build a back end. Some pages are totally fine being static html. If your site is gonna have more than like 3 pages, at the very least, you'll probably wanna use some type of templating system though. That just allows you to have one header, footer, sidebar, etc. and change only the content in the middle of the page on every page load. Again, I used Go's html/template library but you can acomplish this same thing using straight up raw PHP and using includes. Handlebars is a good option too.
5. Front end frameworks are so much more trouble than they're worth
I kinda already explained this in point number 2 but yeah, unless you're building a social media platform or some kinda giant corporate dashboard, you probably don't need React.
6. Be sure to handle 404s and catch-all routes
Ok, so here's where things get a bit interesting. I do have to admit, one thing about using frameworks is that they have a lot of built in stuff that handles 'edge-cases' like I'm about to talk about real quick. People, bots, whatever trying to access stuff on your site that they shouldn't (AKA 'hacking'). And when I first launched my site, I didn't really think too much about handling 404s. I did some handling for people trying to access songs or blog pages that didn't exist, but I didn't think about stuff like them trying to see what I have inside "/uploads" or if they could pull up "/wp-login.php" or whatever and try to guess my password. Now, the way I initially had this set up was that if anyone tried some shit like this, they'd just get my home page every time. And I was cool with that... until I realized that Google was penalizing me for it. See... I was getting crawl errors in the Google Search Console that were saying I had duplicate pages. For a while, I was confused cause I was like "I don'thave any duplicate pages, wtf..." Then one day I was like "OOOOOOHHHHHHHHH" it thinks all those non-existent paths are duplicates of the home page!!!!!! So yeah, I just updated my code so ONLY the "/" route would serve the homepage (could be "/index.html" or whatever if you're site is using that), and that fixed the problem. So yeah, small thing but HUGE impact on whether Google hates you or not.
7. Set up a fall-back static page to avoid showing a '502 Bad Gateway' page while doing deployments. Docker takes a bit of time to go up.
I honestly didn't think about this til recently. I don't really get all that much traffic, so I usually just do my deployments at night. For those of you who don't know, a 'deployment' is just when you push your latest code changes to go live. If you've got a static website, it pretty much happens in less than a second (another plus 1 for static HTML). Just 'git pull' then boom, your server has the latest code. Buuuuuut, since I'm using Docker and there are some pretty big packages that have to be installed and 'built' every time I deploy, it takes some time to run a project after I pull the latest code from Git. In my case, about 3 minutes. Now, like I said, I was just doing my deployemnts late at night when I was least likely to be having traffic... or just being like F it, I'm doing it in the middle of the day! Who cares haha. But this means, every time I do that, there's a solid 3 minutes where if someone goes to my site, they just see a big ugly error message that says "502 Bad Gateway". Even though it makes a site look broken as shit, I was cool with that for a while... but then I remembered Twitter's 'Fail Whale'... and I was like, let me just update my server routing so that I can fall back to some static html page when I'm doing deployemnts! Probably should have implemented this earlier, but ya know. Some stuff is just lower priority until you get other stuff going. There are even better ways to handle this with even less downtime but for now, I think this is pretty good for now.
8. If using a CSS framework, use purgeCSS to strip out unused classes
So, I didn't realize this until I started running Google Page Speed tests on my site but I had a bunch of CSS classes that exist in the CSS framework I'm using (Bulma) that I was not even using. Like A LOT. Just slowing my site down for no reason. So I found this npm package called purgeCSS. It basically scans all of your html to see which CSS classes you're using and deletes the ones you're not. This saved a few MBs!!! Which is huge for slow connections.
9. You probably shouldn't even use a CSS framework at all
To the above point... that all made me stop and ask myself, "Why am I even using a CSS framework in the first place?", to which my answer was pretty much... "Because, this is what we've always done." I remember on my first Web Dev job, and I had this guy who was a kinda experienced Web Designer and he was kinda helping me get set up and showing me around and stuff. Then at one point, he asked me: "So, which CSS framework do you use?" To which I replied, "None, what's that? I just write the CSS all by hand." And he gave me this really confused look that, to this day, I can't tell if it meant, "Wow, impressive" or "How did you get this job?" but since then, (well actually the next job after that) I always used one because I just thought it was the right thing to do. I remember at one point being on Bootstrap, which was notoriously bloated, so I switched to ones like Base CSS, then ultimately Bulma and stayed there until now. But honestly, I really only use it for the grid. I'm using the buttons too, but those will be replaced with some custom designs soon. So that got me thinking... Instead of doing all these acrobatics to use parts of the framework, then strip out other parts, and replace some with custom styles and all that... Why don't I just got back to writing only the CSS that I need... By hand? So yeah, I haven't quite done that yet but it's next on the agenda since I'm about to do a complete re-design soon. Plus CSS has had native grid support for a while now, so I'll just use that. Or maybe even go back to tables haha.
10. Cache all your assets in the Nginx config
Pretty self explanatory, but something I neglected at launch. At the Nginx or Apache level, just make sure your server is automatically caching stuff that never really changes. It speeds up subsequent page loads A LOT for users with slow connections and saves you a bit on bandwidth.
11. Create a route that generates a sitemap.xml and html version
Also self explanatory. I kinda thought that in the age of 'AGI Almost Bro', crawlers were intelligent enough to automatically follow all the internal links on your page, but alas, they are not. You still need to give them a table of contents. Of course, if your site is a static site, you can just write the xml by hand or use some tool to generate it for you, but if you're creating a back end, just create a route that spits it out. Then, any time you add new content, it'll just be there.
12. Set up automatic submission to 'Index Now' to submit your new content to Bing
Yeah, yeah, I know... nobody uses Bing, but hear me out! We need all the traffic we can get! So yes, I'll take the Bing crumbs too please! They have a thing in their Web Master Tools dashboard that lets you submit urls to it (in addition to them also wanting a sitemap, I know... kinda greedy), but they also integrate with this new thing called 'Index Now', which let's you do it programatically. So, in my case, I added some code to my 'add blog post' and 'add song' functions so that whenever I publish new stuff, the url to it immediately gets submitted to IndexNow. So I don't have to log into Bing and submit it manually. Not necessary, but it definitely saves some time so I can focus on making the actual MUSIC.
13. Use something like FFmpeg or ImageMagick to automaticlly format media files for you
I pretty much already described this, but I've created sites like this in the past where I did all the re-sizing, compression, and conversions manually and it gets old fast. So like, if your site is an art site (music, photography, drawings, whatever) it's great to be able to just upload your final high res pieces and have them automatically processed.
14. Set up a cron job that does periodic backups of your database and uploads! (or do it manually every now and then)
If you're managing everything yourself, you MUST set up some way for you to get backups of your stuff. Sometimes things go wrong and when they do, it's kinda necessary to like... not lose everything. Of course, your site's source code will be backed up in the Git repository (You are using Git, aren't you?) and on your local machine, but the content uploaded to it and everything in the database exists separately from that. So you need to be sure to make backups. For my site, I have some code that runs once a day and takes a snapshot of my database and all the files in my uploads folder and shoots them over to to an AWS S3 bucket. If you don't wanna get Jeff Bezos involved in your project, you can just manually save these backups to a usb or external drive too.
15. Set it up so that database migrations run on app startup to prevent having to run them manually
If you're using some kind of SQL based database, you're going to have to run migrations to update your database whenever you make changes to the schema. You can do this manually. I actually did for a while, but it got old fast, so I just put a little bit of code into the back end so that when it starts up, it checks to see if there are any migrations that haven't run yet and runs them. Saves time especially on deployments because you don't have to worry about that awkward time between having to first bring up your app to have it running (if using Docker), then having it be broken because the database queries don't work until you run the migrations. On another note, this is one thing that NoSQL databases like MongoDB have going for them over SQL. I'm not sure about all NoSQL, but Mongo doesn't require any migrations. Because each 'document' can be different from eachother, you can just edit the schema any time you want during development. That does make dealing with the db a lot easier.
16. Make sure all pages have the right meta tags, including canonical links, etc
It's easy to kinda forget about this. I didn't realize until I went to share my music page one day and realized the social media apps shared it as the home page. At first, I thought they were tripping, then I went and checked my meta tags and realized I had the same 'canonical' tag set for that page as the home page. So yeah, check through all that stuff and make sure it's right for each page. Make sure you have meta, Open Graph, and Twitter, tags. They have these online tools called like, Open Graph tag testers or something. They'll tell you what needs to be fixed.
17. Use Google Pagespeed to see where your performance weak points are
Self explanatory. Just run your site through that and it'll tell you what you need to fix to make it perform better. If you live somewhere where there's always fast internet, you can easily get spoiled and and fooled into thinking your site loads as fast for everyone else as it does for you. This could not be further from the actual truth. Some people are on really slow connections, and your site could be taking like 10 seconds to load for them.
18. Use Google Analytics to track your metrics and see what your traffic looks like
Self explanatory. You can roll your own analytics of course. I actually added some really basic stuff to my site that just tells me how many hits each page gets, but honestly that was just for vanity metrics so I could show a page hit counter in the footer. But it's probably a waste of time to build out a full fledged custom analytics solution unless you're dogmatic about de-Googling your existence, which, if you are... I can dig it.
19. Use Google Search Console / Bing Webmaster tools to see how well your site is optimized for Search Engines
A lotta people say SEO (Search Engine Optimization) is dead because of AI and people's general loss of interest in surfing the web without social media... but Idk. I still get some traffic from search engines and at the very least, if someone does Google you, (or Bings you), you do wanna at least pop up. So just use these tools to make sure you're not doing anything to shoot yourself in the foot.
20. Move heavy JavaScript to the end of the HTML doc unless it's needed for rendering something important
Google Pagespeed will tell you this, but off the rip I can tell you, just put the JavaScript into the site's footer instead of the head. Any time you see some instructions for some type of JavaScript plugin, they always tell you to put it in the 'head' tag of your html. With the exception of Google Analytics, this is usually bad advice. Having it there will block the rest of your site form rendering until it finishes running. 9 times out of time, it's fine to load all the HTML and CSS first, then run JavaScript. But yeah, you might wanna have the Analytics tag in the header cause it'll give you more acurate data on if someone 'bounces' off your page or not since it'll run before they get a chance to click out.
21. Add the robots.txt file
I can't believe we're still doing this in 2026 AD, but it seems like there's still some merit to it??? Idk, just add it, probably.
22. If using Docker you MUST use Docker Volumes if you plan on storing stuff to an 'uploads' directory or having a database.
Docker containers only live 'in memory', so they'll disappear when your container goes down. And it WILL have to go down when you deploy updated versions of your site. So get familiar with Volumes so you don't make the mistake of getting things up and running and seeing them go bye-bye when you re-start the server.
23. You probably don't need Captcha on public forms (at least not until you're popping)
I took a risk and just put forms on my site (mailing list, contact, post comments, guestbook, etc.) withought requiring logins, or captchas or anything, and yeah, I get some bot submissions, but it's extremely minimal. Now, I might regret this decision at some point in the future, but it really is less of a deal than I even thought it would be. If I ever start getting flooded with spam, I'll just lock it down then.
Conclusion!!!
So these are the main things I'd keep in mind if I were doing this all over again from scratch. But if I could do it all over again, I'd do it the exact way I did. I'm pretty happy with it. Like I mentioned above, I can do whatever I want with it. Plus I can keep building it out over time. I'm actually working on an e-shop right now. It's not ready yet but it'll be live in a few weeks. It's not gonna be too crazy, just T-shirts to start off with. Then I'm also gonna get some CDs and maybe Vinyl pressed up, but we'll see about all that. Also, I probably forgot some stuff so I'll update this post if I think of anything else. When I drop the e-shop, there's gonna be a lot more lessons learned but I'll probably make a separate post for that.
