Minimum requirements of a good engineering employee

As title, just some random thought…

  1. Able to get things done independently (should be something reasonably complex and takes days or weeks to finish)
  2. Care what he is doing and be responsible to his products
  3. Good understanding of the current progress of the project and able to communicate this with the team
Just to be complete, maybe I should also add 0,
0. Able to vision ahead and create useful tools and accomplish tasks that were not requested.
I guess 0 is more advance but is still essential for mid/upper level staff.
Advertisements

Debugging Torch with Zerobrane

I am kind of new to Zerobrane studio and followed this guide to try to debug Torch code. However, I came across with the error

attempt to index a nil value (global 'utils')

for

printf = utils.printf

It turns out that the path was not setup correctly. In the config file (Edit | Preferences | Settings: User), instead of something like

path.torch = [[/home/user/torch/install/bin/]],
path.torch = '/home/user/torch/install/bin/th'

works for me.

 

Problem upgrading Tensorflow under Anaconda

I tried to upgrade tensorflow today. I have a virtualenv for version 2 installed under anaconda 3. If I simply run

pip install -U tensorflow

I got error like

Cannot remove entries from nonexistent file /home/user/dummy/anaconda32/envs/tst/lib/site-pack
ages/easy-install.pth

It turns out that this appears to be a bug from Ananconda, adding “–ignore-installed” seems to be able to workaround the problem. That is,

pip install -U tensorflow --ignore-installed

 

 

 

Xournal for presentation

I am using more and more beamer this year. Since I used linux, I used to use evince to display my slides. But in last class, I would like to write something on the slides and I was not prepared for that. My laptop is a nice good old X230t with a stylus on it. I can open xournal to write on the slides. But when I made it fullscreen, I could not get away with the toolbars previously.

It turns out to be very easy. Following this post, one can just modified ~/.xournal/config by adding

interface_fullscreen=drawarea

It will then hide the toolbars when switch to fullscreen (F11).

A simple voting example with node.js

Backstory

It started that I would like to have my students to vote for their peers’ presentations. And I don’t like to do it the old fashion paper way. I thought of using some commercial products. But since I was reading a little bit JavaScript couple weeks ago, I guess it probably would be a nice small project to hone my skill. It spent me two days to put together the pieces but it turns out the final thing is quite short and simple.

Before I started, I only knew a little bit of JavaScript and just realized that node.js was Google’s reply of JavaScript server-side programming. I have primitive understanding of server-client models and I knew some html but hardly expert on that also. I knew little to none web programming (I did some php more than a decade ago and I literally forgot everything). I am an electrical engineering (signal processing guy) by training. I am good at matlab, use some python/numpy, and know some java (don’t write it much tho). That’s about it.

I will try to go through my experience of learning the things. But if you just want to try it out yourself, you can skip right to here.

Form and Formidable

The first thing I wanted was some kind of a form that I can let my students submit their votes.  After some google search, I settled with this example with the Formidable. The basic form is like

var formidable = require('formidable');

function processAllFieldsOfTheForm(req, res) {
 var form = new formidable.IncomingForm();
 var user_id = req.param('id');

form.parse(req, function (err, fields, files) { ... // fields, files 
              // contain dictionary structure of the form input

Great! It looks super simple and I can get all information after I hit the form submit button. That is exactly what I want.

Express-Formidable

I can’t help coming across with “express” after a bit more online reading of node.js.  It is actually the most widely used package and its bare minimum definitely looks simple

var express = require('express')
var app = express()

app.get('/', function (req, res) {
  res.send('Hello World')
})

app.listen(3000)

What I need though is how I may combine express with formidable, I am sure someone did that before. I came across this express-formidable package but couldn’t quite make that work. It turns out that it seems that I can just use (require) the two packages together without much issues. Anyway, problems (kind of) solved.

MySQL

Next, I need to store my students’ votes. Relational database should be sufficient. MySQL seems a reasonable choice for me. I got some help from this post.

var mysql = require('mysql');

...

var pool = mysql.createPool({
 connectionLimit : 100, //important
 host : 'localhost',
 user : 'root',
 password : '12345678',
 database : 'votes',
 debug : false
});

function handle_database(req,res) {
  pool.getConnection(function(err,connection){
  if (err) {
   res.json({"code" : 100, "status" : "Error in connection database"});
   return;
  }

  console.log('connected as id ' + connection.threadId);
  connection.query("select * from votes",function(err,rows){
   connection.release();
   if(!err) {
     res.json(rows);
   } 
  });

  connection.on('error', function(err) { 
    res.json({"code" : 100, "status" : "Error in connection database"});
    return; 
  });
 });
}

The above code looks a bit messy. But there is only just one statement pool.getConnection inside handle_database. And the connection.query function is simple enough to grasp. The rest is sort of boiling plate code.

Vote again?

The rest of my journey was mostly spent on how to check if my student has already cast his votes. I want to let them know they have already voted and ask if they want to overwrite their previous votes. I initialized their votes as null and so it is not difficult to make an SQL call to see if he or she has voted before. My biggest problem tho was how to alert them for the change. I initially would like to use the confirm box. Then I realized that I can’t do that because I am programming at the server side. Since I don’t know much web programming at all, the simplest thing I can think of is simply to create another page with a form to ask for confirmation.

Something as simple as the following should suits the purpose.

 <html>
 <body>
 <H2>Are you sure to overwrite your votes?</H2>
 <form action="" method="post" enctype="multipart/form-data">
 <fieldset> 
 <input type="submit" name="action" value="Update" />
 <input type="submit" name="action" value="Cancel" />
 </fieldset>
 </form>
 </body>
</html>

 

Passing previous form parameters

My bigger problem though is the parameters from previous form is now lost. And if my student clicks confirmed, I don’t know what votes I should update to. After several hours of struggling, I settled with an imperfect workaround.Ignoring security issues, I will just pass the votes that my student just cast as get parameters.And when he confirms, I will just read the parameters again from post. Okay. It probably is a bad solution but it kind of works for my simple toy application.

// sending vote information back to browser 
res.redirect('/confirm?name='+fields['name']+'&presenter1='+presenter1+
 '&score1='+fields[presenter1]+'&presenter2='+presenter2+
 '&score2='+fields[presenter2]);
...
// read parameters from browser again
 var name = req.param('name');
 var presenter1 = req.param('presenter1');
 var presenter2 = req.param('presenter2');
 var score1 = req.param('score1');
 var score2 = req.param('score2');

 

Debugging

It was reasonably smooth. But since I didn’t read any documentation and just compiling stuff from what I guessed it meant, I did came across quite a bit trouble with res.send, res.end, res.write, and res.writehead. Basically, I just used them without knowing that I should follow some particular ordering rule. If you have similar troubles, just use res.write all the time and finish it with res.end. res.send is essentially the same as the combination of above and thus cannot be called more than once.

Putting things together

Setting up MySQL

I am using ubuntu and setting up mysql is very easy. Just run

sudo apt-get install mysql-server

and you will be asked for root password. I just used 12345678 for this example.

Preparing MySQL database and table

I will need to first make up a database containing a table to store my students’ votes. To make it simple, I use one column for each student voter and one row for each speaker. And I pre-constructed the entire table with the score first set to null. In command line, run

mysql -u root -p

and input 12345678 for password. Then use the following to create the database and table.

create database votes;
use votes;

CREATE TABLE votes
(
 id INT unsigned NOT NULL AUTO_INCREMENT, # Unique ID for the record
 speaker VARCHAR(150) NOT NULL, # Name of the speaker
 aakash float, # votes by aakash
 ahmad_a float, # votes by ahmad
 ahmad_m float, # ...
 dong float,
 muhanad float,
 naim float, 
 obada float,
 siraj float,
 soubhi float,
 tamer float,
 varun float,
 PRIMARY KEY (id) # Make the id the primary key
);

insert into votes (speaker) values ('aakash'),('ahmad_a'),('ahmad_m'),('dong'),('muhanad'),('naim'),('obada'),('siraj'),('soubhi'),('tamer'),('varun');

Setting up and start node.js

It is very easy to setup nodes.js in ubuntu. I can just install with apt-get install directly

sudo apt-get install nodejs
sudo apt-get install npm

I can then update it with

sudo npm cache clean -f
sudo npm install -g n
sudo n stable

I should then setup and install required packages

npm init
npm install express --save 
npm install formidable --save
npm install mysql --save

Once you put all required files to the current folder. The server can be started with

node test2.js

and it will be listening at port 3000. Try it on your browser with http://localhost:3000.

Final words

It is my first node.js project and I am not a web developer by trade. I do host my own simple website but have skipped the entire web 2.0 thing. Overall I am quite impressed with node.js. It looks like that with some imagination, one can build lots of thing easily even for amateurs like me.

Why I use Linux

This is a re-post of a blog post I wrote for my department blog couple years ago. I just read it again and I found it still relevant.
For those of you who have been to my office or have taken my classes will know that I am a big Linux fan. I started to use a little bit of Linux since early 2000. But I was “converted” completely around 2007 or 2008. Since then, Linux is my main OS. But first things first, I want to emphasize that I am not an MS hater. I used most of the MS OSes before, from MSDOS, Win3.x, Win95, Win98, WinME (yes,it is a horrible one), Win2000, WinXP, Vista, and Win 7. (Sorry, Win 8. I don’t see that I may own any more new Windows device soon in the future. And Mac, yeah, I know you are cool. But everything I need is offered by Linux already. I am not sure I want to pay the premium for you.)
Even though I only started using Linux as the main OS of my computers for the last several years, I have been exposed to the Unix-like system quite a bit earlier. Back in the days of early 90’s (yeah, I am old, at least quite a bit older than many of you reading this) when I was an undergrad student, workstations were popular and we had to telnet to some mainframe computers to do programming assignments. And interestingly, the Unix commands I learned then are almost identical to the Linux commands I use now.
I guess you see why I am writing all this… because while I see some of you are Linux experts, it just came to me that many of the students I met here have little exposure of Unix-like systems. Of course, I myself am more or less a user only rather than a realexpert. But I think that some experience to Unixlike system is essential to any electrical/computer/tel ecom engineers. I can think of many reasons for that. Just name a few,
1). Linux is ubiquitous. It is used by different devices, from very small embedded systems, to enormous mainframes and super computers.
2). Linux is free. And so do many tools on its platform aswell. Need to typeset your paper? Use Latex/Kile. Need to edit your photo? Use GIMP. Need to dosome math? Use Octave. Of course, some of this software is available on Windows as well. But it is often way easier to install them on Linux than Windows because most of them were developed on theformer natively.
3). Linux is reallyfree. Not to mention that the commonly used application software is free on Linux. Some very sophisticate or specialized software is free there also. Just name a few, ns-2/3 for network simulation, zoneminder for security system, and musescore for music composition and notation. And you can even setup a professional (and completely free) web server at home with Linux in an hour or so.
4). Your future boss wants you to know it. Because of the above, there is a good chance that your future bosses would like someone to at least familiar withthe OS. So being acquaint with Linux will allow you to access to many more possible positions in the job market.
If the above reason is not sufficient for you to start playing with Linux, here is another one: using Linux/Unix-like system makes one more productive. Okay. It sounds a bit subjective and I am not sure if anyone does any statistical study for that. But there are indeed quite a few productivity features that Unix-like system has offered for decades while one is still waiting for MS to catch up. The most noticeable one is virtual desktop. I just can’t understand whyMS still doesn’t implement that one. Another very useful productive feature is packaging systemsfor software installation. Okay, MS finally got this one eventually and we have “apps for windows”… But when I say Linux can make one more productive. The first thing came to my mind are not the above. What is more important to me is that Linux comes with lots of useful built-in command line tools. Yes. I mean old fashion command line tools. I know everyone likes GUI and command line stuffs seem scary and difficult to use. But GUI tools tend to become obsolete much faster than command line tools. As I mentioned earlier, I am still using the commands such as sed/grep/cut/find/tr that I learned 20 years ago. And those commands were introduced like 40 years ago. So your time learning those things won’t be wasted soon in the future. In contrast, every Office version looks a bit different and it takes time to adapt a new one!
To be fair, Windows provide some batch commands also. But they are less powerful and more importantly, far less portable outside the windows world.In contrast, you can run those Linux commands with little modification on a Mac right away. Besides, one can use these tools on Windows too if they really want to. For example, by installing a Linux layer, such as Cygwin, on top of windows. But honestly, it just makes life complicated.
Actually, not just command line tools are available on Linux, everything needs to be done can be done with them alone. By design, GUI tools on Linux are really just wrappers for these “primitive” commands. Therefore, one can always fall back to these commands and get things done especially when resources are really limited. For example, at this right moment, I am connecting to my office desktop using ssh even though I am in a time zone 13 hours away from Tulsa. If I had a windows box, the remote connection would definitely be unusable as the GUI stuffs would have sucked up so much resources that the poor connection here just couldn’t afford it.
 1  Okay, Cygwin may seem like a godsend for Windows veteranswho want to start playing with Linux. But frankly, I would not recommend Cygwin to anyone Linux newbies. Cygwin is far less user friendly than modern Linux distros like Ubuntu. One canencounter way more problems installing and running any software on Cygwin than on distro like Ubuntu.
For those who are still not convinced, let me illustrate the power of Linux with an example task that I do occasionally. From time to time, my collaborator generates many figures in eps format but we actually need them in pdf format instead. And often we have the files scattered in several subfolders (say, under the current folder) and I have to convert all the files. Imagine that if someone were using Windows, first he or she would need to figure out how to convert eps file to pdf format as there is no built-in tools for that. Maybe someone eventually found some software (most likely not free) but then he would still need to find a way to convert all of them efficiently. If he was very lucky, maybe there was a batch conversion function offered by the software. But at many times, he would have ended up wasting an hour more converting the files one by one manually even after paying like $20 for the software.
On a contrary, if you use Linux and are knowledgeable with some Linux commands, you are really lucky in such situation. All the hassle can be done in a single line:
find . -name "*.eps" -print | sed 's/\(.*\)\.eps/ps2pdf "\1.eps" "\1.pdf"/' |sh
The command seems complicated but is actually very easy to understand. It is really just a cascade of three commands linked by the pipe character “|”. The firstcommand, find . -name “*.eps” -print, simply search for all eps files under the current folder including all the subfolders. The second command, sed ‘s/\(.*\)\.eps/ps2pdf “\1.eps” “\1.pdf/”’, tries to replace each eps file, say with a name ‘XXX.eps’, by a command string ‘ps2pdf “XXX.eps” “XXX.pdf” ‘. And the third command, sh, simply executes the command string created in the previous step.
Okay, this definitely is getting too long and I probably should stop here. And this finishes my advice (rant) of why all engineering students should know (and use) Linux. Actually, I also believe all graduate engineering students should use Latex (instead of Words) for their papers. But maybe I will write about it some other time.