Sunday, August 25, 2013

Automation of Grunt builds in WebStorm/PHPStorm

It's very handy to use Yeoman and Grunt to build AngularJS (and not only) applications. And I'll write here how make it even more comfortable in JetBrains IDE.

Default installation of Yeoman is very good for development on localhost. But it's not always possible - some databases and services just can't be on localhost, even their mocks. For example: ActiveDirectory - I don't even want to imagine, how many time I'll spend to make small copy of AD on my localhost.

For deployment I use beanstalk - all I need to deploy on development server is press "git commit and push" button in IDE (I use PHPStorm because in my current projects backend is in PHP).
But before that, I have to run "grunt build" to put all results to the "dist" folder.

There is 2 ways to automate grunt builds in PHPStorm/WebStorm:

1) File Watchers - can be used, when repository contains only 1 application (because of "working directory" argument in the File Watcher settings);
2) Rebuild manually by running external tool from context menu or executing "grunt build" in console;

To run grunt build in Windows console:

1) add path to nodejs folder and %APPDATA%\Roaming\npm\ in your system PATH variable
2) open console in IDE (shift+ctrl+x)
3) if necessary, set working directory by "cd dir_name"
4) write "grunt.cmd build"

To create context menu "Grunt build" in Mac OS:


Settings -> External Tools -> +
Name: build
Group: Grunt
Program: grunt
Parameters: build
Working directory: $FileDir$

To create context menu "Grunt build" in Windows:


Settings -> External Tools -> +
Name: build
Group: Grunt
Program: grunt.cmd
Parameters: build
Working directory: $FileDir$

Now Grunt will rebuild our project, but it's not yet enough - names of some files will be changed and we'll
need to add these files to git (run "git add .").

To do this, I use grunt-exec. Run in apps folder:
npm install grunt-exec
then add to Gruntfile.js (before last '}' for instance):
grunt.loadNpmTasks('grunt-exec');
then create 'exec' task in grunt.initConfig:
exec: {
  git_add:{
    command: 'git add .',
    cwd: '<%= yeoman.dist %>'
  }
},
now, to run this task in each build, add this line in list of
grunt.registerTask('build',
section. It will look something like that:
  grunt.registerTask('build', [
    'clean:dist',
    'useminPrepare',
    'concurrent:dist',
    'concat',
    'copy',
    'cdnify',
    'ngmin',
    'cssmin',
    'uglify',
    'rev',
    'usemin',
    'exec'
  ]);
That's all! :)

Wednesday, July 24, 2013

How to install Golang 1.1 in Debian from repository

I like to use packages in Debian - it's easy to update and more stable.
Version of Go language in Debian Wheezy is 1.0.2, but we have 1.1 version in unstable (sid) branch, so let's use it.

Add sources list

It's not safe to just add "unstable" into sources list, so I chose to add it in this way: http://serverfault.com/a/382101/82650 , I'll quote text of Brendan Byrd here just to save it:

First, create the following files in /etc/apt/preferences.d:
security.pref:
Pin: release l=Debian-Security
Pin-Priority: 1000
Package: *
stable.pref:
Pin: release a=stable
Pin-Priority: 995
Package: *
unstable.pref:
Package: *
Pin: release a=unstable
Pin-Priority: 50
Now, creating a matching set for /etc/apt/sources.list.d:
security.list:
deb http://security.debian.org/   stable/updates  main contrib non-free
deb     http://security.debian.org/  testing/updates main contrib non-free
stable.list:
deb     http://ftp.us.debian.org/debian/    stable main contrib non-free
deb-src http://ftp.us.debian.org/debian/    stable main contrib non-free
unstable.list: Same as stable.list, except with unstable.

Install

Then run apt-get update and, finally:
aptitude install golang/unstable

Check

Check version:
go version

And check code execution like described in http://golang.org/doc/install#testing

Thursday, July 18, 2013

PHP, MSSQL, nvarchar (fetch and write UTF-8 with ODBC)

Today I found pretty annoying thing in PHP MSSQL ODBC driver (I use it via PDO).
This driver doesn't support nvarchar fields (or nvarchar(max) only, doesn't matter for me - I can't change database schema just because of using PHP).
After 2 hours of googling I found solution. It's hack, but it works fine with nvarchar, decimal, varchar, int and others.

To read 

I found it accidentally in this blog and I'm very thankful to David Walsh:
SELECT CAST(CAST([field] AS VARCHAR(8000)) AS TEXT) AS field FROM table

To write

It's from StackOverflow:
$value = 'ŽČŘĚÝÁÖ';
$value = iconv('UTF-8', 'UTF-16LE', $value); //convert into native encoding 
$value = bin2hex($value); //convert into hexadecimal
$query = 'INSERT INTO some_table (some_nvarchar_field)  VALUES(CONVERT(nvarchar(MAX), 0x'.$value.'))'; 

Wednesday, June 12, 2013

Google recommendations about multi-lingual sites are wrong

I think recommendations from Google Webmaster Tools - Multi-regional and multilingual sites are bad. I'm just one person, humble programmer, and I dare to think that I'm smarter than Google? Sounds too brave even for me, but in this case - yes.

Google recommends use sub-domains or query path part as definition of current language, so URL will look like https://lang.example.com or https://example.com/lang/.
And there is reasons why I think it's bad recommendations:

1) When user will share link, language will be hardcoded.

It's not obvious when your main language is English, but if user is, for example, from Italy and wants to share some URL from Wikipedia in his favorite social network, all users, from all countries, will open that page in foreign language and will be forced to search where is the language-switching button on that site. Try to find that button on Google Webmaster Tools site.

2) It's bad for semantic web.

Water is a water. In France, Italy, Germany and even in Soviet Russia, water is H2O. So by URL wiki.org/water (it's example) I suppose to read about water. Will de.wiki.org/water or wiki.org/fr/water point to different resource, not water? If not - why different URL?

3) Language is just an option of content representation, same as background color.

Main thing is the meaning of that content, not color or language or rounded borders. So use URL parameters, which you use for other options. It's not intuitive to use us.example.com/api/ and in.example.com/api/ - is it different APIs? Will their responses be different only in language or in some other things also? Time in response will be in PDT or in IST?

But it's Google recommendations, source of visitors...

Of course it's enough reason to follow these recommendations if you want to optimize your site as much as possible for free visitors from search engines. But I'm sure that sites should be for humans, not for search engines. And I hope some engineers from Google will read this (or similar) article and will change their recommendations.

Sunday, January 20, 2013

Couchbase support in Jamm!Memory

New storage

Jamm!Memory, PHP library with universal interface for best cachers (Redis, APC, Memcached), now have object for work with Couchbase - CouchbaseObject.
All features of this library are implemented in this class: tags, dog-pile and race-conditions protection, lock/acquire and others.
All code is test-covered, tested and even benchmarked. CouchbaseObject is not yet used in production, but I'm going to (see below).

Main feature of this storage is a fault-tolerance, of course.
While working on this code and testing it, I built cluster from few nodes and it was really simple. Server GUI is super user-friendly, has lot of monitoring things and control of nodes. Add new node in cluster is just  2 minutes (maybe less) plus time for rebalancing (it's going automatically and all data is available).

And next branch

Age of Jamm!Memory API is more than 2 years already. So I made branch 1.0, where currently placed all code of library, tested by unit-tests and by heavy usage in production for 2 years.
In next branch, 2.0, I will write refactored version of Jamm!Memory. API will be changed, all features will be saved, new features will be added. Some other storages maybe will be added (Riak, Cassandra).

Plans

I want to create another library, dedicated to Redis Cluster. It's in active development still, but it's time to write clients already :) It will be only key-value storage, but I hope it will as fast as existing Redis. I don't like that "Redis cluster sacrifices fault tolerance for consistence", but maybe it's just me and I don't understand something.

Also I hope Couchbase will have ability to turn off permanent writes to disk and will be able to copy data in background, so all writes and reads of data will be in memory. 
Please, support this idea by posting in forum thread.

In their forums I've created few themes and accidentally (I swear) whole column "last post" had become filled with my nickname :)

Also, I'm going to use Couchbase in production as a storage for cache, logs, sessions. This DB could be perfect for analytics (because of Map/Reduce), but it will require to write some GUI to show all these results. Currently used tool works only with RDBMS by ODBC. 

Saturday, January 19, 2013

Installing Couchbase PHP SDK for PHP 5.4 + Couchbase Server 2.0

  1. Couchbase Server doesn't have repositories (at this moment). That's not cool, of course. Forget about apt-get update.
  2. Precompiled SDK package for PHP is not compatible with PHP 5.4 API. So don't waste your time and compile it from sources. And, of course, forget about apt-get update. Very "enterprise" way, what can I say...
  3. Couchbase Server GUI looks very attractive. Has email alerts and lot of graphics for monitoring.
  4. No hot-fixes for Community version.
If in this article you are looking instructions "how to install", then just open pages with instructions:

Server:
Couchbase Server Manual - Installation
Don't forget to configure server in web-interface.

Client:
Install PHP SDK for Couchbase
At the moment of writing this article, precompiled PHP SDK package is not compatible with PHP 5.4, so use sources (link "Source archive."). 
Before compilation, install packages php5-dev and php-pear. 
Then download, unzip, go to unzipped folder and run 
phpize
./configure
make
sudo make install
and copy path of generated module from output. You need to add it into php.ini
For php5-fpm it will be:
sudo nano /etc/php5/fpm/conf.d/couchbase.ini
and add line (use generated path + "couchbase.so").
extension=/usr/lib/php5/20100525+lfs/couchbase.so
Now restart php
sudo /etc/init.d/php5-fpm restart 
And check it:
php -m | grep couchbase 
In output you should see "couchbase". Also, in phpinfo() you will see something like
Now you can write your genius code :)
For auto-completion in IDE you can use https://github.com/couchbase/php-ext-couchbase/blob/master/example/couchbase-api.php

Friday, January 18, 2013

Reading Couchbase documentation

In this post I want just highlight some wondering lines from Couchbase documentation

Couchbase

"All Couchbase SDKs automatically read and write data to the right node in a cluster. If database topology changes, the SDK responds automatically and correctly distribute read/write requests to the right cluster nodes. Similarly, if your cluster experiences server failure, SDKs will automatically direct requests to still-functioning nodes."

"For optimal performance and speed, stores most frequently used data in RAM. Persistent storage also provided to survive system downtime and for re-population of RAM."

"For example, a player making 10 game state mutations in 5 seconds, such as planting 10 flowers in 5 seconds, will be compressed by Couchbase automatically into just one queued disk mutation."

"To extend our application with new user attributes, we simply start storing additional fields at the document level. Unlike traditional relational databases, there is no need for us to have server downtime, or database migration to a new schema."


"You may wonder how effective it is to run query your view if Couchbase Server will run it on every persisted document in the database. But Couchbase Server is designed to avoid duplicate work. It will run the function on all documents once, when you first query the view. For subsequent queries on the view Couchbase Server will recompute the keys and values only for documents that have changed."

Impression

Finally I've read that Couchbase Developer's Guide 2.0
Very good example of documentation. Not the best, but very good. Many DB sites can learn from Couchbase how to write good documentation.

In some articles too many words and trivial examples (even with pictures), but sometimes amount of examples in live code is just not enough. For examples, after reading whole manual, I still don't know how to create "view" and how send it to server. Some file saving... But I know how to write functions in view and even best practices about it :)

Of course, I'm very impressed about Couchbase at all.
Ability to write map/reduce functions in JavaScript is amazing.
Auto-updating results of view is awesome.
Finally I know how my idea about select via callback function should be named - "map function" - I knew I'm reinventing the wheel :)

Tomorrow

I'm planing to add support of Couchbase to Jamm!Memory, I'm planing create there branch 2.0 (for "cas" and some other things in all cachers and to rename some methods to more familiar get/set).

I'm going to learn Ruby and try to write project in Ruby. Examples in Couchebase documentation shows that Ruby can be readable even easily than PHP.

I really think that Couchbase is the best in the NoSQL world at this moment. But maybe I just don't know all of them :)