There's no two ways about it; putting together a website is hard! You need to know a myriad of programming languages (HTML, PHP, CSS, AJAX, Javascript, MySQL, just to name a few). You also must understand how to combine these tools into one cohesive whole, which is hopefully more than the sum of its parts. These days, we use frameworks, boilerplates and generators, but do you actually - when you get right down to it - know how to build a website from scratch?
This tutorial is meant for those who can write a PHP function, know what jQuery is, can handle their CSS well, but would like some insider info on how a web service is built from scratch. How should I organize my files, where should I keep my functions, how should I start planning the thing, how should I connect to the database, how should I handle AJAX calls and how should I manage my 404 pages are just some of the questions I try to shed some light on here.
Disclaimer
Before we begin, a quick disclaimer: while we are building a functioning service, it does not take huge measures to ensure security and scalability. Basic things like password encryption, .htaccess rules, permission checking are included, but there is much more you can and should do if you want to create something for mass consumption.
The app is so far from perfect, but I thought it would be a refreshing change of pace to show you the "How I get there and plan to continue" process instead of the "Here is something awesome I've developed". Web development is not a one-cycle job where you build something great in step 1 and money rolls in at step 2 (it would be awesome though). You usually iterate through versions of a project, arriving at something unique and great along the way.
I hope to demonstrate the underlying logic of some methods you can use so you can make up your own mind on how to do things. I tried to point out the errors and bad practices (some of which I used) along the way. I encourage you to question as many steps as possible and think of better ways of doing things, maybe even dropping off your own ideas, via the comments.
Introduction
The service we will be creating is called Bonsai Writer, a tool which will convert plain text into HTML. It is aimed mostly at writers but it can also be used as a note stash for more general use.
Personally, I write articles for many magazines, all of which have different styling guidelines. However, when I write, I want to focus on writing not styling. The idea of Bonsai Writer was born, a tool for me to manage my articles easily.
Step 1: Planning our Application
Planning should be the very first thing you do and should take quite a long time. I built Bonsai Writer on a whim with only 20-30 minutes planning and it shows! It works and looks satisfactory, but with a few extra hours it could be much better!
Project Planning
The first step in any project should be the planning of the various stages your project would go through. This can be as elaborate as a Gantt diagram, spanning multiple resources and sub-projects, but it can be as simple as a list and a few words about what you want to do. I suggest planning in proportion to your application. If you are undertaking a six month project, it will do you good to create some documents, spreadsheets and other tools to assist you. For a project as simple as this, however, an ordinary list of goals should suffice.
Wireframing
Wireframing is the process of creating a quick low-fidelity mockup of your website.
As the first step, I I drew two quite basic wireframes on my whiteboard: one depicted the front page, the other the app itself. I highly recommend whiteboards as planning tools. I have a 5x2.5 foot board on my wall, and it helps me tremendously in the planning stages.
Some basic wireframes to point me in the right direction.
As you can see, this is all rather simple, but it gave me a solid point for moving on.
If you are developing an app for wide usage, I suggest spending much more time on wireframing. I have found that the more you wireframe, the faster you will build your website. So even though you are not coding, you are still on your way. A good wireframe could contain more views, indication of what happens when you click an important link, some text explaining what we're looking at and so on. If you'd like to learn more about wireframing, I suggest taking a look at Wireframes Magazine.
Designing
When working on project I like to follow up the wireframing with designing key frames. The website really only has three views so I created rudimentary designs of the front page and app page. I had a solid idea in my head for the error page, so I decided to skip that. This is not great practice. Getting things down on 'paper" can be a huge help. Apart from the fact that you can share it, it also helps to provide you with more perspective. I've many times kept a design idea in my head for ages thinking it was awesome, then, once drawn, it looked really horrible after a few days.
The resulting designs
As you can see, I elaborated on the app wireframe in the design. While this is okay here, in general, it is best not to do so. If this would have happened on a large project, I would have switched back to wireframing to determine the best structure. In an ideal world, I would have thought of this in the initial wireframing process anyway, underlining how important that step is for all future parts of your website.
The design process itself was quite simple. I knew I wanted a nice bonsai tree, so I began by finding a nice image on Photodune. I found a great image of a Maple Bonsai and, after some cropping, cutting and the application of one of the filters, I created the tree you see on the site now.
I decided to go for a brushed pastel feel, so I settled on a peachy color, creating a sort of monotone design. It's fairly obvious that I am no designer, but it will do just fine for out purposes here.
The Coding Plan
So based on the above, what are we building here? It helps me to think of a project from three different angles.
- On the database level. we will need a table for storing articles and users, so it might help to think of a good database structure.
- On the backend, we will need to be able to retrieve and save article and convert plain text to HTML and be able to log users in and out.
- For the frontend, we want to enable users to switch between views (plain text and HTML) with key combinations; we want them to be able to show and hide the menu with keystrokes; we need some sort of lightbox for some popups and a few other minor things.
If you want to go into more detail, you can draw up your database schema, write some skeleton functions, decide what password encoding to use, how to log users in and out (sessions or cookies for example), decide if you want to use a JS framework like jQuery and so on. You can gather all the external resources you need in advance for quick and easy implementation when you get there.
On my end, I also set up a version control system right away, SVN, along with a bug tracking system Redmine. I plan on making this app much better in the future so version control and bug tracking will be useful tools to have. I find that both are of great use on personal or non-collaborative projects as well. Version control gives you great development history (not to mention a backup), and the bug tracker is an execllent place to put bugs you want to fix, ideas you have and other notes.
Step 2: Preparing the Application
Where are we developing?
Now that we know roughly where we're going, let's get cracking. Depending your development environment, some minor basics will need to be modified. The most common setups are:
- Using localhost on your computer and using a subdirectory (eg: http://localhost/bonsaiwriter)
- Using localhost and a virtual host (eg: http://bonsaiwriter)
- Using a remote host with a dedicated domain (eg: http://mybonsaiwriter.com)
- Using a remote host with a subdomain (eg: http://bonsaiwriter.mysite.com)
- Using a remote host and a sub directory (eg: http://mysite.com/bonsaiwriter)
We will create a config file so we can cater for most of the differences there, and I will point out other issues as they arise. For now, make sure you are aware of which method you're using; don't worry, we can make the app work either way.
Creating a basic file structure
I started out by creating some of the files I knew were going to be needed, and placed them in their respective directories. This step enables me to mentally picture how the website will be structured and visualize how files are included into one another as the website runs. Here's what I came up with after a first pass.
- .htaccess
- index.php
- config.php
- load.php
- header-app.php
- header-site.php
- header-error.php
- footer-app.php
- footer-site.php
- footer-error.php
- 404.php
- style.less
- [DIR] app
- index.php
- [DIR] website
- index.php
- [DIR] js
- [DIR] images
- [DIR] includes
- functions.php
- appFunctions.php
The idea is that the "point of entry" file is always the root index.php file. Using .htaccess rules, all requests are sent to the main index.php file and that file determines what to do with them. This essentially means that no matter what you type in after "bonsaiwriter.com/', the main index.php file will be loaded.
To make sure that we can always use our database connection, function and other defined bits and pieces, we include load.php into every single request using another .htaccess rule. This loader file with pull in the config.php file, set up any variables, make functions we need available and so on - more on this later.
Depending on the page we are on, different files will be included via index.php. If we are looking at the app, the header-app.php file will be included, then the app/index.php file, then the footer-app.php file from the main directory. You may be wondering why the header and footer files are not in their respective directories. The primary reason is for future proofing and ease of use. In reality, the error header is not just a header for errors, it is a header for all those pages which are structured like an error page. It would be more appropriate to name that file any-page-which-just-has-the-tree-and-some-text-header.php. In our case, this is just the error page but later on this may evolve so the file is kept in the root dir to make sure I can easily include it into any file later on.
The js and images directories are empty for now, but I think are rather self explanatory. We will put all our js files into the js directory and all our images into the images directory - not exactly rocket science.
The includes directory will hold any function files or third party files we want to include. The first in there - functions.php - contains general functions which are needed for the site itself to work. A function for displaying the menu or creating an excerpt would be found in here. The appFunctions.php file will include any app specific functions for pulling the data of an article, saving it and so on.
We will manage our website's looks with a .less file.
LESS is a dynamic stylesheet language which allows us to add some programming logic to our CSS, and make working with styles a much cleaner experience.
The use of LESS will require us to reference a Javascript file as well, but this is okay because it provides us with so much flexibility. In a production environment, I would compile the LESS file to a proper CSS file, minify it and use that, but for this project, we will keep things as is.
A badly drawn diagram of how the loader works.
Setting up our loader
I first wanted to put the loader logic in place. The desired outcome is that, when you visit the root URL (http://bonsaiwriter.com in my case), the following things happen.
- load.php fires up
- config.php is included into this file right away to set things up
- our functions are included (functions.php and appFunctions.php)
- a database connection is made
- index.php is finally loaded
To get things going, head into your .htaccess file and add the following line:
php_value auto_prepend_file load.php
If you are working online, in some cases, your host (for example, Bluehost) won't allow you to include files, via .htaccess. Close and repoen your .htaccess file after trying to load your page and you should see that the auto prepend line has been commented out. If this happens, create a php.ini file in the root directory as well and use the following code in it:
auto_prepend_file = "load.php"
Next, open load.php, and type, "Hi." Open your index.php file and type ", you are awesome." If you load the website in your browser, you should see "Hi, you are awesome" being shown. This verifies that index.php is being loaded and that load.php is prepended to it. Next, let's set up our load.php file properly using the code below.
<!--?php <br ?-->/**
* The Website Loader File
*
* This file is responsible for loading the whole website.
* It includes the necessary files, fires up the database connection
* and so on.
*
* @author Daniel Pataki
* @version 0.1 alpha
* @package bonsaiwriter
*/
// Load the configuration file
include("config.php");
// Load functions
include(INCLUDES_PATH."/functions.php");
include(INCLUDES_PATH."/appFunctions.php");
// Create a database connection
// We will add this in soon
?>
As you can see, I have added documentation to this file using the phpdoc syntax. This won't just be useful if other developers work on the project, but for you as well. If you return to it in two months to fix a bug or two, you will have no idea what's going on anymore. I will leave out the documentation on the following examples but they are there in most files; take a look at the project pack for more information.
Within load.php, the first thing we must do is include our config file, then the two function files. I have used a constant to store the path which is defined in config.php (see the next section). The last thing is establishing a database connection, which we will get to once our config file is in place.
Configuring our App
In order to transfer the app to a different domain and have various frequently used data at our fingertips, it is advisable to create a configuration file. In the first iteration, I defined a handful of constants.
// Site paths and urls
define("PATH", "/home6/tastique/public_html/bonsaiwriter");
define("SITE_PATH", PATH."/site");
define("APP_PATH", PATH."/app");
define("INCLUDES_PATH", PATH."/includes");
define("URL", "http://bonsaiwriter.com");
define("AJAX_URL", URL."/ajax.php");
// Database Details
define("DB_NAME", "bonsaiwriter");
define("DB_USER", "bonsaiwriter");
define("DB_PASS", "supersecretpass");
define("DB_HOST", "localhost");
At this stage, I didn't really think things through so it may well be that we won't use APP_PATH at all, but it seemed like a good idea at the time - we can always delete it later. Never be afraid of refactoring.
These two blocks (paths and db info) will enable you to work on your localhost and then transfer the site to an online host in a matter of seconds. All you need to do is modify the values here.
Connecting to a database
The next stage in our preparation is connecting to a database. First, let's make sure we have created one. If you are using localhost, go into PhpMyAdmin (http://localhost/phpmyadmin, or http://localhost/phpMyAdmin usually) and create a new database by typing its name (bonsaiwriter) into the "Create a new database" field. Go back to the main PhpMyAdmin screen, and click the privileges tab. Click "Add a new user" below the table and use "bonsaiwriter" for the username, "localhost" for the host and "supersecretpass" for your password (although you might want to try a more secure one). Next to "global privileges," click the "check all link" and click go. You have created a database and a user which has all the privileges necessary to work with the database.
If you are online, you might not be able to create a database and a user via PhpMyAdmin. Many hosts restrict the use of this tool and you will have to use a dedicated section in the cPanel or somewhere else to accomplish this. If your host uses cPanel, I suggest using the MySQL Database Wizard to create a database and a user.
Once finished, return to the load.php file, and replace the commented line "We will add this in soon," with the following code:
$bwdb = mysqli_connect(DB_HOST, DB_USER, DB_PASS, DB_NAME);
This will create a new connection for us, and allow us to use the created $bwdb object for out database transaction needs.
Setting up our database tables
Once you have a database set up, you can add tables to it, via PhpMyAdmin. Just go into your database and run the following SQL.
CREATE TABLE IF NOT EXISTS `article` ( `ID` int(11) NOT NULL AUTO_INCREMENT, `uri` varchar(5) CHARACTER SET latin1 COLLATE latin1_general_cs NOT NULL, `raw` text NOT NULL, `html` text NOT NULL, `title` varchar(120) NOT NULL DEFAULT 'Untitled Article', `user_id` int(11) NOT NULL, `date_created` datetime NOT NULL, `date_updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `public` int(1) NOT NULL DEFAULT '1', PRIMARY KEY (`ID`), UNIQUE KEY `uri` (`uri`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 ; CREATE TABLE IF NOT EXISTS `user` ( `ID` int(11) NOT NULL AUTO_INCREMENT, `username` varchar(18) NOT NULL, `password` char(40) NOT NULL, `email` varchar(255) NOT NULL, PRIMARY KEY (`ID`), UNIQUE KEY `username` (`username`,`email`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 ;
This creates the table structure required for storing articles and users. We store things like the plain text and HTML, creation date, etc. for articles, and users consist simply of a username, password and email. Take a look at the created table for more information.
Redirecting all requests to the main index file
Now we're all set: when you load the website everything is prepended, defined, included, connected and so on, but before we move on, I want to add some .htaccess magic to make sure that no matter what URL is entered, index.php is loaded.
The reason for this is because I plan on each article having a URL like this: http://bonsaiwriter.com/8h3ef/ Since I don't want to create directories and index files for all the billions of possibilities I'll just forward all requests to one file and deal with them there. The result of this is that, if you go to http://bonsaiwriter.com/i2e23e/23e23/e23e2/23e23/, you will not get an error page, the contents of index.php will be shown. We will obviously need to remedy this for cases where a page clearly doesn't exist, but more on that in a bit!
To make sure that all your requests are handled by one file, open your .htaccess file and paste in the following lines under the existing prepend rule.
Options -Indexes
RewriteEngine on
RewriteCond %{REQUEST_URI} ^
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)?$ /index.php [L]
This is where things may be a bit different depending on your setup. If you are using a virtual host, a dedicated domain or a sub-domain, this should work. If you are using a sub directory, you will need to modify the last line, like so:
RewriteRule ^(.+)?$ /subdirectory/index.php [L]
Once this has been put into place, you should be able to type anything into the URL bar after your base domain and it all should still be well - you should see index.php being loaded.
Showing the Right Views
Our index.php file will be our "routing" file. It will determine what you want to see and show it to you. It will know that http://bonsaiwriter.com/about/ exists but http://bonsaieriter.com/aboutus/ does not. It will know that there is an article at http://bonsaiwriter.com/334Dr/ and it will show you the app with the article, however it will detect that there is no article at http://bonsaiwriter.com/wr4rD/, so it will display an error page.
Let's look at the pages we will have first, and then determine how to direct users there. Here's a quick breakdown:
- Landing page (http://bonsaiwriter.com)
- Contact, about and legal pages (http://bonsaiwriter.com/contact/)
- Create a new article (http://bonsaiwriter.com/new/)
- Article pages (http://bonsaiwriter.com/r4J2w/)
Now all we have to do is write rules for these cases and make sure that users are sent to the correct places. Let's start by capturing the page we are on, and stripping out the parts of the URL we don't need.
$page = str_replace("?".$_SERVER["QUERY_STRING"], "", $_SERVER["REQUEST_URI"]);
$page = substr($page, 1, -1);
The above code should be placed at the top of the index.php file, and will return the portion of the URL we need. If you go to http://bonsaiwriter.com/about/ it will return "about;" if you go to http://bonsaiwriter.com/this/is/a/page/, it will return "this/is/a/page".
The method above will also need to be modified a bit if you are using subdirectories, but this can easily be done by stripping it out from the URL using the str_replace() function or something similar.
If the $page variable is empty, we are on the main page. If its value is "new," we want to create a new article. If it is contact, about or legal we want to show the respective pages. In all other cases, we will check if an article exists at the given URL. If it does, then we show the article. If it doesn't, we have a 404 error so an error is shown. Let's look at the code which does all this.
if(empty($page)) {
include(SITE_PATH."/index.php");
exit();
}
elseif($page == 'new') {
$uri = generate_article_uri();
$ID = insert_article($uri);
header("Location: ".URL."/".$uri."/");
exit();
}
elseif(in_array($page, $pages) AND file_exists(SITE_PATH."/".$page.".php")) {
include(SITE_PATH."/".$page.".php");
exit();
}
elseif(article_exists($page)) {
include(APP_PATH."/index.php");
exit();
}
else {
include(PATH."/404.php");
exit();
}
Firstly, we check if the $page variable is empty. If it is, we include the index.php from our site folder, since we want to show the front page.
If the $page variable is "new," we want to create a new article. To accomplish this, we generate a unique article identifier, henceforth called a uri. We then insert a new article into the database with that uri and redirect the user to the article's page. We will discuss these specific functions further down.
The next section deals with our predefined pages. I added a simple array to config.php, which contains the pages that we want to have:
$pages = array("contact", "about", "legal");
If the current page is amongst these, and the file to present the page exists (site/pagename.php), we include that file, otherwise we move on to the next condition.
If the code gets to this next block, we have determined the following:
- We are not showing the main page
- We do not want to create a new article
- We are not showing any predefined page
All that is left is either an article page, or we have an error. In this block, we check if we have an article that has a uri which is the same as the data contained in our $page variable. We use a separate function for that which returns true if an article does exist, and false if it doesn't (more on this soon). If the article does exist, we include the index.php file from the app folder.
If none of the above conditions have been met, all we are left with is that this must be an error page so we display the 404.php file.
With that complete, we have accounted for all the possibilities that can happen while viewing our little web app. We've defined the set of pages that can be viewed and have said that all other pages are error pages.
Functions for our App
Let's continue by defining and writing some of the functions that our application will need. I started out with the following six functions - three of which we've used already:
- article_exists - When passed a uri as an argument this function will tell us if it exists
- generate_article_uri - Generates a random and unique 5 character string to identify an article
- insert_article - Add an article to the database
- get_article - Retrieves a single article from the database based on its uri
- save_article - Saves the details of an article
- !convert_plain_to_html - Method for converting raw text to HTML
Checking for an existing article
The function is extremely simple. We pass it a uri, and determine the database to make sure an article with that uri does not exist.
function article_exists($uri) {
global $bwdb;
$results = $bwdb->query("SELECT ID FROM article WHERE uri = '$uri' ");
if($results->num_rows == 0) {
return false;
}
else {
return true;
}
}
We create a result set which consists of all the articles with the given uri. If the number of rows returned is 0, we are fine, the article does not exist. Otherwise, at least one article has been found, so we need to return true - the article exists.
Generating a unique uri
Generating a unique uri is an integral part of the application. Whenever someone creates a new article, it needs to have a unique identifier, but we don't want extremely long urls either and we don't want to take tons of time finding a unique one. Here is how Bonsai Writer does it.
function generate_article_uri() {
global $bwdb;
$allowed = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
$allowed = str_split($allowed);
do {
$characters = array();
for($i=0; $i shuffle($allowed);
$characters[] = $allowed[0];
}
$uri = implode($characters);
}
while (article_exists());
return $uri;
}
First, we define all the characters we will allow. I have allowed all upper and lower case characters and all the numbers. This will give us 42 to the power of 42 number of variations - we should be safe for a while.
I have given all the allowed characters in the form of a string, and then I split it up into an array using the str_split() function. I then created a which will run five times, choosing a random member of the allowed characters array (by shuffling it on each pass and selecting the first value), adding it to our array of chosen characters. I then implode this chosen array to arrive at the final string.
As you can see, all this is in a do-while loop. This works just like a regular while loop, except that the loop is executed at least once, no matter what. In regular while loops the condition is examined first and, if it is true, the loop is run once followed by the condition being examined again. In do-while loops, the loop is run once, only then is the condition examined.
This way we can create a uri, and then check it against the database using the article_exists() function. As long as that function returns true (which means we have randomly generated a uri which exists) the loop will run again, generating another uri and checking it.
It might seem surprising, but this method is less than great if you have a huge volume website. 49 to the power of 49 is a lot, but let's look at how this could be a problem with a smaller set. Suppose you want to assign people 4 digit unique pin codes. This gives you 10,000 variations. Your site is visited by 20 people a day so 10,000 is fine for a while.
In a year's time, about 7,300 pin codes will be taken so you still have ages to implement a new system. However, since 73% of the pin codes are taken, your random number generator will spit out a taken pin code 73% of the time (essentially 2 out of 3 times). If I went to your website, it may take the system several tries before it finds a unique pin. This equates to extra database queries.
Choosing a right scheme here is important and I think with 49 characters available for 5 spots (that gives you 6 with 83 zeros after it options), we really are safe for a while.
Inserting an article
Initially, when someone creates an article, it has no data - just a uri - so inserting it is rather easy:
function insert_article($uri) {
global $bwdb;
$bwdb->query("INSERT INTO article (uri) VALUES ('$uri') ");
return $bwdb->insert_id;
}
I do want to mention an important note about the uri here. Since we specifically allowed upper and lower case letters "abcde" should not be the same as "Abcde". While this is obvious on the PHP level, we need to make sure that the uri field in the database has a case sensitive collation. In my case, all data receives "latin1_swedish_ci" as its default collation which is case insensitive - indicated by the "ci" at the end. For the uri field, I chose "latin_general_cs" to make sure we can check for uris properly.
Getting a single article
When retrieving an article, we simply want to have all its details available to work with so all we need to do is query its row from the database.
function get_article($uri = false) {
global $bwdb;
if($uri == false) {
$uri = substr($_SERVER["REQUEST_URI"], 1, -1);
}
$results = $bwdb->query("SELECT * FROM article WHERE uri = '$uri' ");
$article = $results->fetch_object();
if($article) {
return $article;
}
else {
return false;
}
}
Passing this function a uri is optional. The reason for this is that on single article pages, we usually have a uri available in the url anyway. So if the uri is not given, we just parse it from the url.
We then get the row of that article, putting all the data in the $article variable which we return for use. If an article is not found, we return false.
Saving the article
I originally wrote a function which converts our plain text to HTML and then saves the article. Subsequently I realized that some separation would be better here. It is good practice to have one function to one thing and one thing alone. The saving function should not be responsible for conversion. We might use a conversion function inside the saving functions but if we separate them the point is that both functions are available separately. Luckily I came back to rewrite the article to seem like this is how I planned it all along.
function action_save_article() {
global $bwdb;
$title = "";
$raw = "";
$html = convert_plain_to_html($raw);
$uri = $_POST["uri"];
$bwdb->query("UPDATE article SET title = '$title', raw = '$raw', html = '$html' WHERE uri = '$_POST[uri]' ");
}
The only things I know with certainty is that the uri will be passed to this function using the post method and that the $html variable will get its data by converting the value in the $raw variable. In any case, this is just a placeholder for now; we'll see how things progress as we go along.
Building the Front Page
Now that we have some basic functions ready, let's make something nice. The front page on the wireframe consists of three sections. There's the top section with the tree and the call to action button, the middle section with the video on the left, and a footer with three links. I actually modified this a bit, simply because I don't have time to do the video now, but the top part remains unchanged at the time of this writing.
If you remember from the previous sections, when you load any page, the main index.php file is used. When you look at the front page, it cleverly recognizes your intent and decides to include the site/index.php file. At this stage, there is still absolutely no output involved anywhere. This file contains the get_header() function which outputs the top of our page followed by some content and then the get_footer() function pulls in the bottom part of our page. Right until you call the get_header() function there is no output so you are free to set cookies, session data, output HTTP headers and so on.
The first thing I figured out was out a good HTML header for the website. The following is the top of the header-site.php file.
<!--?php get_head("site") ?-->
This is a HTML doctype with the html tag, a get_head() function and the body tag with the view in the id attribute (in this case site, but it could be app, or error as well). I have chosen to use a function to get all the header data like the site title, scripts, styles and so on because its cleaner and provides you with one more level of flexibility. If you look at the code in functions.php for the get_head() function, all it does is pull in the head-site.php file, so let's go there now and take a look.
Bonsai Writer <!-- Favicons --> <!-- Styles --> <!-- Scripts --> <script type="text/javascript" src="http://lesscss.googlecode.com/files/less-1.1.3.min.js"></script>
There might be some extra bits in there in the project files, but this is the gist of it. We have our title and character set defined, a bunch of different favicons tied to the page, our style is included along with the LESS JavaScript pulled in from Google Code.
To get LESS working, you need to include the LESS stylesheet first and then the LESS script. This will automagically make LESS work as intended.
I won't explain the rest of the HTML here; take a look at the header file and the LESS file to see what's going on. There's nothing special about the HTML involved.
Building the Error Page -
When viewing an error page, the 404.php file will be included. This file works just like the index file in the site directory. It includes a header, has some content, and includes the footer. The only added bit is the setting of a header, which we can do at the top of this file since there is no output at that point yet.
Setting the header is extremely important since, otherwise, the user will receive a 200 header - meaning everything is okay. Code-wise, everything is okay, since we are requesting a file (index.php) which is there, so we do have the resource the user is looking for on a file level. However, we know that we don't have the content, so we need to tell the world that this is actually a 404 page.
<!--?php header("HTTP/1.1 404 Not Found"); ?-->
Since I presume that the app will have a few different error pages in the future, I decided to modularize the next step instead of coding it directly. I created a fatal_error() function which takes two parameters - a title and some content - and display's a nice error message. The function can be found in functions.php.
function fatal_error($title, $content) {
get_header("error");
echo "</pre>
<h1>".$title."</h1>
<pre>
";
echo "</pre>
<div class="error_text">".$content."</div>
<pre>
";
get_footer("error");
}
The function itself pulls in the error header, adds the title and content, then pulls in the footer. The LESS file makes this all nice and pretty, but otherwise we're done with the error page.
This is a great example of bad planning and coding on my part. First of all, I would not consider a 404 error to be a fatal error so the function is incorrectly named. Also, since I intend to use this function for all errors, it would make much more sense to pass this an error number argument and have it generate the HTTP header itself which, right now, is done outside of the function.
If you are working to quickly put up an alpha version or you're building an advanced wireframe, I think these mistakes are okay. We know they are there, we can come and fix them later; we just want to have something online to get feedback and try for ourselves.
Building the App Page
We've finally arrived to the point where we can build our app. I know in advance that I want four distinct things on this page. I want to make a nice header bar, a sidebar, an editor and a preview area. The header will always be visible, the sidebar can be hidden with a key combination or a click on the tree. By default only the editor will be visible, but using a key combination the editor can be hidden and the preview area can be shown.
I first went into the header-app.php file and made sure that the details of the article being viewed are pulled into a variable. Since we wrote a function for this, it won't cause a huge headache, and i will be able to refer to this later on.
global $article; $article = get_article();
I also made sure that jQuery was added to the head-app.php file, pulled from Google Code. I added a file to the js directory named app.js and tied it to the app by adding it to the head-app.php file.
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js"></script> <script type="text/javascript" src="<?php echo URL ?>/js/app.js"></script>
Once done, I went into the app/index.php file and created this skeleton.
</pre> <div id="app_content"><textarea id="editor"><?php echo $article->raw ?></textarea> </div> <pre>
I made sure to create the menu.php file in the app directory and placed some placeholder text in it. I also hid the preview area using my CSS skills. Additionally I added some rudimentary styling to put everything roughly where it was going to be. Since the next thing on my mind was building some JS controls I gave everything clashing background colors so I could see what was going on.
Showing and hiding elements with JavaScript
It's time to write some JavaScript! My first order of business was to go into app.js, write the following into it, and reload the app page.
$(document).ready(function() {
alert("lol");
})
On reload, you should see an alert box with the text, "lol," in it. This verifies that your JavaScript file is tied to the app and that jQuery is working as well. Let's continue by enabling users to control the visibility of the menu using the ctrl+space key combination.
$(document).keydown(function(e) {
if(e.which == 32 && e.ctrlKey == true) {
$("#app_menu").toggle();
return false;
}
})
When using key event functions, you can utilize the parameter of the function to obtain all sorts of information about the button which was pushed. The space key has the keycode of 32, and the ctrlKey attribute will be true if the control was pressed. So if the keycode is indeed 32 and the ctrlKey is true, the user has pushed the ctrl+space key combination. In this case, let's toggle the menu - closing it if it is open and vice versa. We also return false to make sure a space is not inserted into the texture of the editor if we are using it.
If you want to see all the details related to a key press, you can use the following method which works in most browsers which have a JavaScript debugger built in or Firebug installed.
$(document).keydown(function(e) {
console.log(e);
})
Right after I did this, I used some abstraction to separate the toggling of the menu from the event. The two reasons for this are:
- The toggling of the menu will eventually need much more code than just this one toggle line. We will need to change some classes and even send an AJAX call to store the status of the menu later on so it is kept closed throughout page loads if a user closes it once
- The menu can be controlled in other ways as well so we don't want to duplicate code. You can also hide and show the menu by clicking on the tree icon so pasting in the code to open the menu and save its state in this eventuality as well is redundant and unnecessary.
$(document).keydown(function(e) {
if(e.which == 32 && e.ctrlKey == true) {
toggle_menu()
return false
}
})
function toggle_menu() {
if($("#app_menu").is(":visible")) {
$("#app_logo").removeClass('current')
}
else {
$("#app_logo").addClass('current')
}
$("#app_menu").toggle()
}
Let's follow the same logic when creating the action which will switch our view from edit to preview mode using the shift+space key combination.
$(document).keydown(function(e) {
if(e.which == 32 && e.shiftKey == true) {
switch_mode()
return false
}
})
function switch_mode() {
$("#editor").toggle();
$("#preview").toggle();
}
With that done, the main part of our application frontend is complete. All that we need to do is save the data in the text field, convert it and make it available in the preview box. Before we do that, I'll make a slight detour to show you how this app handles AJAX requests, but I'll be back to saving our data in no time.
Handling AJAX
If you're familiar with how WordPress handles AJAX requests, then you are on friendly territory. Otherwise, this method might seem a bit complex compared to the functionality we need. I will explain my reasoning for using this method once I've shown you how it actually works.
All AJAX calls in the app are handled by the ajax.php file in the main directory. This file checks that the AJAX call originates from inside the app, that it has the proper parameters, and so on.
To get an AJAX call to work in this app, you will need to do at least three things:
- Give it the ajax.php file as the target URL
- Make sure it passes an "action" parameter
- Make sure there is a function defined in a function file with the name of the passed action parameter
Let's look at the innards of ajax.php to better understand all this.
if(isset($_SERVER["HTTP_REFERER"]) OR empty($_SERVER["HTTP_REFERER"]) OR substr($_SERVER["HTTP_REFERER"],0, strlen(URL)) == URL) {
$title = "A fatal error has occurred";
$content = "
It seems like you are trying to initiate an action from outside the app which
either means that you are a naughty person, or that a very weird error has occured in our app.
Please <a href="".URL."/contact/">contact us</a> if you think you have found an error with the app
<a class="button" href="".URL."">go to the website »</a>
";
fatal_error($title, $content);
die();
}
if(isset($_REQUEST["action"]) OR empty($_REQUEST["action"]) OR is_string($_REQUEST["action"])) {
$title = "A fatal error has occured";
$content = "
It seems like there is an incorrect link in our application. It very probable that
the action you initiated was not completed. Please <a href="".URL."/contact/">contact us</a>
if you think you have found an error with the app.
<a class="button" href="".$_SERVER[">« go back</a>
";
fatal_error($title, $content);
die();
}
call_user_func($_REQUEST["action"]);
In the first if block, we check the referrer (where we came from) to make sure there is no naughty business. It must be defined, it can not be empty and it must start with our base url. If this condition is not met, we generate an error page. The second block makes sure that an action is defined. If something is wrong, we show an error page again.
As an interesting side note, it was at this point that I realized that I should have defined my error header in the fatal_error() function. I also realized that the first error actually is a fatal error while the second isn't; so it would be better to rename the function.
If all is well, we call the function, which has the name of our action parameter's value. If you pass "save_article" as the parameter, the save_article() function will be run.
In our application, we will only have 1-2 AJAX calls, so we could just create separate files for them. This is a valid opinion but I always try to think ahead. This setup enables me to do much more. In fact, by the time you read this, I may extend it a bit to accommodate for more flexibility. Being able to essentially "plug" our AJAX calls into existing functions makes everyone's lives much easier and is the basis for a plugins system, which may yet pop into existence if the app becomes popular.
Converting plain text to HTML
At the very root of the service is its capability to convert text to HTML. There are a lot of factors which go into this and a lot more will be developed, but let's look at some of the most common conversions we will need to pull off. We will use a lot of regular expressions to parse the plain text for symbols, characters and whatnot to help us convert it to HTML.
Saving the data
As the first step, let's make sure that we are saving the changes without converting them. Let's add some JavaScript, which will save our article after every 30th key press.
k = 0
$(document).keyup( function() {
if(k % 30 == 0) {
save_article()
}
k++
})
function save_article() {
raw = $("#editor").val()
uri = $("body").attr("data-uri")
$.ajax({
url: ajaxurl,
type: "post",
data: {raw : raw, uri : uri, action : "action_save_article"},
success: function(response) {
$("#preview").html(response)
}
})
}
First, we define the variable k which holds the number of keys pressed since the page was loaded. We add one to that variable each time a key is pressed. If the variable can be divided by 30 without a remainder we save the article.
The save_article function grabs our plain text from the editor and the uri of the article which is stored as an attribute of the body tag. We then make an AJAX call with the help of jQuery. We set the url to ajaxurl - a variable defined in head-app.php - we set the type to post and define the data we want to send. This data includes the plain text, the uri and an action which is needed for the AJAX call to work; this is the function which will be run. We then take the response of the PHP script we run and output it to the preview box.
We previously created a skeleton for the action_save_article() function in the appFunctions.php file; so let's go there now and modify it.
function action_save_article() {
global $bwdb;
$raw = $_POST["raw"];
$uri = $_POST["uri"];
$tpos = strpos($raw, "\n");
$title = substr($raw, 0, $tpos);
if (empty($title)) {
$title = "Untitled Article";
}
$title = $bwdb->real_escape_string($title);
$body = substr($raw, $tpos);
$html = convert_plain_to_html($body);
echo "</pre>
<h1>\n".$title."\n</h1>
<pre>
\n".stripslashes(stripslashes($html));
$bwdb->query("UPDATE article SET title = '$title', raw = '$raw', html = '$html' WHERE uri = '$uri' " );
}
function convert_plain_to_html($plain) {
return $plain;
}
For ease of use, I have pulled the raw text contents into $raw and the uri into $uri. I then found the position of the first line break. Everything before this is the title, everything afterwards is the content. We find the title by using the substr() function and store it in the $title variable. In case it is empty, we make sure we still have a title - "Untitled Article". Finally we make sure to escape the value. This is user input so we need to protect against malicious code being executed on our servers.
The body of the text is everything except the title; so we use substr() again to put the body text into the $body variable. For now, the value of our $html variable is set to be the same as the value of the $body variable. I have created a separate function which will take care of this but for now it just returns the plain text back. We will change this in a second, we just want to make sure this all works. If you type at least 30 characters into your texture and reload the page, the saved text should be pulled into the textarea from the database.
Converting the data
Now its time to modify the value of the $html variable to get the actual HTML. When planning out how this should work, I laid down three guidelines for myself.
- The basis of converting the text into HTML will be the new line character. I will split the whole text into chunks separated by new lines.
- I will convert these chunks to whatever is needed (a list element, a pre code block, etc.), everything else will be converted to a paragraph.
- These chunks can contain additional markup for inline elements like links, code elements, bold text, etc.
With that in mind, let's give this a go. First, let's separate our text by the new lines, and make each chunk created a paragraph.
function convert_plain_to_html($plain) {
$html = explode("\n", $text);
foreach($html as $key => $paragraph) {
if(trim($paragraph) == "") {
unset($html[$key]);
}
else {
$html[$key] .= "
\n".$paragraph."\n
\n";
}
}
$html = implode("\n", $html);
$html = str_replace("\n", "
", $html);
$html = $bwdb->real_escape_string($html);
$html = str_replace("
", "\n", $html);
return $html
}
Initially, we use explode to split our text into an array. Each line will be in a separate member in the array. Now let's go through each member of the array and add some paragraph tags. If the line is empty, we unset that member since we don't want to have empty paragraph tags messing up the flow of our document. Otherwise we add a paragraph tag before and after the text.
Notice that I have added newlines before and after the text and after the closing paragraph tag. This is to ensure that the source code looks nice. Instead of having paragraphs one after another in the code, they will be a bit more manageable. Later on I will add tabs as well to make this all indented. This is not essential for our app to work but in many cases authors will have to paste the source code into the WordPress backend for example, so having it nicely presented is a great bonus.
Our $html array now contains proper HTML so we implode it to make it a string. To make sure no malicious code is sent to our servers, we need to escape the value of the variable but if we do that, our new lines will be escaped as well. To prevent this from happening I temporarily replaced all "\n" instances with something unique. We then escape the string and replace our temporary placeholder with newlines again. With that we are done, we can return our new HTML code.
Replacing inline level elements
Each block level element can contain a number of inline level elements such as links or code. Let's take a look at how we can handle links. First of all, we need to make a rule which will be the method user create links with in the plain text version. For example users could type the following to create a link
<a href="http://thelink.com">This is the anchortext</a>
To parse this out of the text we can use the following code which should be placed in convert_plain_to_html() function inside the foreach loop, before we convert the paragraphs.
preg_match_all("/\<a>([^</a>]*)/", $paragraph, $match_links, PREG_SET_ORDER);
foreach($match_links as $match_link) {
$link = explode("|", $match_link[1]);
$link = "<a href="".$link[1]."">".$link[0]."</a>";
$paragraph = str_replace($match_link[0]."]", $link, $paragraph);
}
Before we start this regular expression match, the value of $paragraph is a line of raw text. In this line of raw text, we want to parse out all these links and convert them to actual HTML links. We are using preg_match_all() because we want to catch all occurrences (there may be more than one link in a paragraph). The first argument is the regular expression, the second is the subject within which we are searching. The third argument will hold our match data and the fourth is a flag which modifies how the matches are returned. See the documentation for some more info on all this preg_match_all.
To match this pattern, all we need to do is search for any occurrence of "[Link:" and make sure to grab everything until we get to a closing bracket. We iterate over all the resulting matches and parse out the anchortext and the url, using explode. The first member of the $link array will be the anchortext, the second the url. We then build an HTML link from these two elements. Finally we search for the original link text in the plain text string and replace it with our shiny new link.
I also wanted to make sure that inline code can be placed in the following way.
You need to use the <code><code here></code> tag to create inline code
Once again, let's do some parsing and replacing. You'll want to place the code below, just after or before the previous link checking code.
preg_match_all("/@@([^@]*)@@/", $paragraph, $match_codes, PREG_SET_ORDER);
foreach($match_codes as $match_code) {
$paragraph = str_replace($match_code[0], "<code>".htmlentities($match_code[1])."</code>", $paragraph);
}
We parse out those bits which start with a double at sign, then contain anything except an at sign, then end in a double at sign. To ensure that code is shown properly, we convert all the appropriate characters into HTML entities and wrap the code tags around the thing. As I am making my final review of this article I have noticed an error in the above code, but I'll leave it in as a good example of how to arrive at better code through iteration.
Normally, it is good practice to search for a string beginning with a specific character and grab everything except some character. This is the method used for HTML tag parsing and other uses, however, it is incorrect in our case. The above regular expression would not play nicely with inline code which contains an at symbol while there is no reason it shouldn't contain one. In our case, the above code should be modified like so:
preg_match_all("/<code>(.*)</code>/", $paragraph, $match_codes, PREG_SET_ORDER);
Converting Block Level Elements
Converting block level elements is a bit trickier because they require an opening and closing tag on a separate line. If we want to create a list, the markup in plain text would look something like this:
Let's create a paragraph here, and then a list underneath. # This is a numbered list # This is its second member # And this is its third member Another paragraph will be written here
It is not enough to detect that one of the lines starts with a hash. We also need to add an opening list tag before the first occurrence and a closing tag after the last element. The way I pulled this off is all right, but it could be much more modular to help me expand on it later.
If we get to a list element, before we do any parsing we should add an opening list tag. Since the next list element is a new line we would be adding opening tags in front of every list element. So in addition to adding the opening tag I also set a globally available variable - let's call it $prev_ol - to true. Id we get to a list element and $prev_ol is false we know we just arrived to a new list so we add a start tag, add the list element and set $prev_ol to true, indicating that we are now in a list. Once we get to the next list element the value of $prev_ol is still true so we don't add an opening tag, we just add a list element. Once we get to any element which is not a list element we make sure that if $prev_ol is true, we add a closing list tag and set $prev_ol to false. Our resulting code is the following.
function convert_plain_to_html($plain) {
$html = explode("\n", $text);
$prev_ol = false;
foreach($html as $key => $paragraph) {
if(trim($paragraph) == "") {
unset($html[$key]);
}
else {
// Parsing inline elements
preg_match_all("/\<a>([^</a>]*)/", $paragraph, $match_links, PREG_SET_ORDER);
foreach($match_links as $match_link) {
$link = explode("|", $match_link[1]);
$link = "<a href="".$link[1]."">".$link[0]."</a>";
$paragraph = str_replace($match_link[0]."]", $link, $paragraph);
}
preg_match_all("/@@([^@]*)@@/", $paragraph, $match_codes, PREG_SET_ORDER);
foreach($match_codes as $match_code) {
$paragraph = str_replace($match_code[0], "<code>".htmlentities($match_code[1])."</code>", $paragraph);
}
// Parsing block level elements
if(isset($match_ulist[1]) AND !empty($match_ulist[1])) {
$html[$key] = "";
if($prev_ol == false) {
$html[$key] .= "</pre>
<ol>
<ol>\n";</ol>
</ol>
<ol>
<ol>}</ol>
</ol>
<ol>
<ol>$html[$key] .= "
<li>\n".$match_olist[1]."\n</li>
</ol>
</ol>
<ol>
<ol>\n";</ol>
</ol>
<ol>
<ol>$prev_ol = true;</ol>
</ol>
<ol>
<ol>}</ol>
</ol>
<ol>
<ol>else {</ol>
</ol>
<ol>
<ol>if($prev_ol == true) {</ol>
</ol>
<ol>$html[$key] .= "</ol>
<pre>
\n";
}
$html[$key] .= "
\n".$paragraph."\n
\n";
$prev_ol = false;
}
}
}
$html = implode("\n", $html);
$html = str_replace("\n", "
", $html);
$html = $bwdb->real_escape_string($html);
$html = str_replace("
", "\n", $html);
return $html
}
As you can see, the code is rapidly becoming more complex here. We also needed to modify how paragraph tags are handled. Since something is either a list or a paragraph at this point, we need to make sure that if we switch from a list to a paragraph the list which is before it is closed. If we are creating a paragraph and $prev_ol is true (we were handling a list before this paragraph), we need to add a closing tag before we start handling our paragraph. We then need to make sure $prev_ol is set to false again.
-- More replacements --
To see more replacements, check out the full function in project files. I have catered for only a handful of possibilities, but I am adding more and more. Most replacements can be categorized into one of the two methods I've described above so adding them should not be a huge problem.
- Enabling user accounts -
Enabling users for this website was a rather easy process, although this is mainly because of the restricted feature set. The three main areas on Bonsai Writer to handle users are registration, authentication and login detection.
Registration
Registration works with a simple form which allows users to submit their email address for registration. This is done via an AJAX call which fires the action_register() function, let's take a look.
function action_register() {
global $bwdb;
$_POST["email"] = trim($_POST["email"]);
if (!isset($_POST["email"]) OR empty($_POST["email"])) {
echo 'please enter an email address';
}
elseif(!filter_var($_POST["email"], FILTER_VALIDATE_EMAIL)) {
echo 'please use a valid address';
}
elseif(check_user_exists($_POST["email"])) {
echo 'that email address has already been registered';
}
else {
if($_POST["notify"] != 'true' ) {
$plain_pass = substr(sha1(time().$_POST["email"]."saltword"),0,8);
$pass = sha1($plain_pass);
$subject = "Welcome To Bonsai Writer";
$message = "</pre>
<h3>Hello and Welcome to Bonsai Writer!</h3>
<pre>
You can now use the password ".$plain_pass." to log in.
We strongly advise changing this when you first log in.
We wish you a nice and productive stay, if you have any comments, suggestions or
problems, feel free to contact us at any time.
Bonsai Writer
";
$headers = 'MIME-Version: 1.0' . "\r\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";
$headers .= 'From: Bonsai Writer ' . "\r\n";
mail($_POST["email"], $subject, $message, $headers);
}
$email = $bwdb->real_escape_string($_POST["email"]);
$bwdb->query("INSERT INTO user (email, password) VALUES ('$email', '$pass')");
if($_POST["notify"] == "true") {
header("Location: ".$_SERVER["HTTP_REFERER"]."?notify=true");
}
else {
echo "success";
}
}
}
First we make sure that the email is given, it's valid, and is not used. If all these tests are passed, we drop the user an email with a password and insert him/her into the database.
The password is generated by creating a string which consists of the current time, the user's email address and an additional secret word. This could be something like this: [email protected]. The time at the beginning, coupled with the email address and a random word gives us a pretty random base. We then pass this through the SHA1 algorithm. At this stage the purpose of this is not to encrypt the password, we just want something random looking as the plain password. We take the first 8 characters from the resulting string, this will be the user's actual password, something like "aXe938Sk". We run this string through the SHA1 algorithm, this time with the intent of encrypting it. We only store the encrypted password for security of course.
Authentication
Whenever a user logs in a form is filled out giving us the email and password of the user. The parameters are passed to the action_log_user_in() function which makes sure the details are correct. It does this by grabbing the email from the form and running the given password through the SHA1 function. It takes a peek at the database to see if a user exists with the given email/encrypted password. If a user exists we put the user details into a session variable. If there was a problem we return an error.
Login state detection
In the load.php file, I included a section which takes a look at the user's session variables. If $_SESSION["login"] exists - we set this in the authentication process - we presume the user is logged in and we pass the details to the $userdata variable. We also have a function is_logged_in() which checks the value of the $userdata variable and returns true if the user is logged in, false if he/she isn't.
A better system
The login system is also somewhat rudimentary and not unhackably safe. I would prefer to use more secure $_COOKIES or some newer methods available, but this was good as a quick placeholder to test functionality. There is no indication here of any standardization to accommodate for additional login methods such as oAuth or integration with services such as Twitter and Facebook.
Wrapping up the Website
There were a number of other techniques utilized in this app that I haven't covered here but you should be able to find all of them by looking into the files provided. For example, the CSS font-face property is used to embed three fonts to make the website look nice, CSS is used to create the round icons at the top bar in the app and so on.
Colorbox, an excellent jQuery plugin is used to create the modal windows, which appear when you click on the icons, and Elastic is used to make our textarea grow dynamically as we type into it.
A wonderful .htaccess ruleset is in the htaccess file which I retrieved from Perishable Press. This will help keep evil hackers out of your site.
A great number of features can be added as well. This is just the initial incarnation - a test version if you will - of this application. It will support custom CSS embedding, the definition of custom rules and many handy functions to help authors and editors do their job. My goal is to have an instant preview of an article in its native environment (whichever online website I am writing for).
Final Thoughts
While this is in no way an extensive article on how to create a web service or a website, I hope it has provided some insight on the workflow and the logic behind building something for the web. Remember, you don't have to get things right in the first round. You just need to be able to make it better and better!
The block level replacement handling is needs refactoring, but not to worry, I will get in there and overhaul it in the next few weeks to make it leaner and the app will be that much better. This iteration of the app will enable me to test the functionality, find the flaws, gauge the requirements of users and make it better bit by bit. The underlying logic of how I do things will not really change, however I will probably add more planning to the next release.
In this tutorial we've learned how to put a rudimentary website together which is a skill all in itself. I see a lot of people who know PHP, HTML, CSS, Javascript; the only thing they are missing is the ability to put it all together on their own. Keep in mind that this is only one way of doing things. In many cases, it is better to go with popular frameworks and pre built templates, but it is always nice to know how to do it yourself.
Comments