Untitled document instead of Google Docs list: how to get your document list again

Until yesterday, the shortcut docs.google.com listed the files I’ve on Drive.

Today when I try to access the domain I can only get a blank, Untitled document.

In that case you have to move the cursor on the title and then click on the arrow.

back2gd

You’ll be back on drive.google.com.

How to enable gzip on proxy servers on Apache

I’m starting to use the gunicorn django app using supervisord. Here my configuration:

  • Varnish: port 80
  • Apache: port 8080
  • gunicorn: port 4180 (/path/to/my/manage.py run_gunicorn localhost:4180)

Only the port 80 is exposed to other clients than localhost. The Varnish default backend is Apache (localhost:8080). I have a Drupal installation and a django installation on the same machine: since I want to expose django on the same domain at a defined location, I add to Apache this location:

<IfModule mod_proxy.c>
ProxyRequests Off
ProxyPreserveHost On

<Proxy *>
Order deny,allow
Allow from all
</Proxy>

# on port 4180 gunicorn is running
# @see /etc/supervisor.conf
ProxyPass /foo http://localhost:4180/
ProxyPassReverse /foo http://localhost:4180/

<Location /mypath>
Order allow,deny
Allow from all
AddOutputFilterByType DEFLATE text/html
</Location>
</IfModule>

You can omit AddOutputFilterByType DEFLATE text/html: here I just take the response from gunicorn, compress and then serve to the client in this way:

(client) -> varnish -> apache -> gunicorn

(client) <- varnish <- apache (compress) <- gunicorn
                (X-Varnish-Cache: MISS) 

Here an example of what I get:
Image

It’s a big page, but using gzip from 2.2 MB of the uncompressed page I get 417 KB gzipped text/html, less than 1/4 of the original!

Autolaunch a command and restart it on quit on CentOS

I’ve a django-admin command running as a server thanks to gevent. I want this server to run on boot and autorestart on quit.  StackOverflow give me a hint: use Supervisor.

On a Centos 5 distro:

# find supervisor for your distro...
yum search supervisor
# ...and install it
yum install supervisor.noarch
nano /etc/supervisord.conf

At the end of the file, add a new program:

[program:myfunnydjangocommand]
command=/usr/bin/env python26 /usr/local/etc/django-apps/foo/manage.py tcpapi 4114
priority=999                ; the relative start priority (default 999)
autostart=true              ; start at supervisord start (default: true)
autorestart=true            ; retstart at unexpected quit (default: true)
; startsecs=-1                ; number of secs prog must stay running (def. 10)
; startretries=3              ; max # of serial start failures (default 3)
exitcodes=0,2               ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT             ; signal used to kill process (default TERM)
; stopwaitsecs=10             ; max num secs to wait before SIGKILL (default 10)
; user=root                   ; setuid to this UNIX account to run the program
log_stdout=true             ; if true, log program stdout (default true)
log_stderr=true             ; if true, log program stderr (def false)
logfile=/var/log/myfunnydjangocommand.log    ; child log path, use NONE for none; default AUTO
logfile_maxbytes=1MB        ; max # logfile bytes b4 rotation (default 50MB)
logfile_backups=10          ; # of logfile backups (default 10)

Then, start supervisord.

service supervisord start

Take a look to supervisord log file:

less +G /var/log/supervisor/supervisord.log

You’ll see something like this:

2013-06-07 11:54:16,559 CRIT Supervisor running as root (no user in config file)
2013-06-07 11:54:16,576 INFO /var/tmp/supervisor.sock:Medusa (V1.1.1.1) started at Fri Jun  7 11:54:16 2013
        Hostname: <unix domain socket>
        Port:/var/tmp/supervisor.sock
2013-06-07 11:54:16,645 CRIT Running without any HTTP authentication checking
2013-06-07 11:54:16,654 INFO daemonizing the process
2013-06-07 11:54:16,657 INFO supervisord started with pid 19316
2013-06-07 11:54:16,666 INFO spawned: 'myfunnydjangocommand' with pid 19318
2013-06-07 11:54:17,670 INFO success: myfunnydjangocommand entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Read documentation about the configuration options but keep in mind your Supervisor version. I don’t use supervisorctl because of this bug, if you get an error simply go with service supervisord… but if you have a newer version this should be already fixed.

Note: myfunnydjangocommand.log doesn’t contain anything useful in my experience but maybe it’s related how I write the output since I’ve written it to use interactively, outputting lines directly to the user. I’ll update this post if I find how to solve this issue.

Django and Drupal integration using drush via SSH

Some months ago I talked about how to achieve a unified login from Django to Drupal using drush. The basic assumption was that both Drupal and Django are on the same server. What if the two components are on different servers?

Paramiko is a SSH2 protocol library aimed to provide simple classes to make SSH connection. Let’s see how the code to call drush on command line changes.

Prerequisites:

  • paramiko
  • on your app settings.py add:
  • DRUPAL_SERVER_SSH_HOST     = '0.0.0.0' # Your host here
    DRUPAL_SERVER_SSH_USERNAME = 'YourRemoteServerUserHere'
    DRUPAL_SERVER_SSH_PASSWORD = 'YourRemoteServerPasswordHere'

    And then:

    assert request.user.drupal_id > 0
    # user id to log in
    drupal_id = str(request.user.drupal_id)
    output = ""
    try:
     # a list with command as first element and arguments following
     get_password_recovery_url = ["drush", "-r", settings.DRUPAL_SITE_PATH, "-l", settings.DRUPAL_SITE_NAME, "user-login", drupal_id]
     # via ssh http://stackoverflow.com/a/3586168/892951
     ssh = paramiko.SSHClient()
     # add to known_host the remote server key if it's not already stored
     # @see http://jessenoller.com/blog/2009/02/05/ssh-programming-with-paramiko-completely-different
     ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
     ssh.connect(settings.DRUPAL_SERVER_SSH_HOST, username=settings.DRUPAL_SERVER_SSH_USERNAME, password=settings.DRUPAL_SERVER_SSH_PASSWORD)
     ssh_stdin, output, ssh_stderr = ssh.exec_command(" ".join(get_password_recovery_url))
     output_lines = output.read().splitlines()
     # taking only the first line of the output:
     # e.g. 'http://example.com.it/user/reset/16/1369986816/67k7ReHi97FdtRfdrrXGqqesyz6FXyy7T8jqHiXxsrY/login'
    except:
     # @todo additional statements here
     pass
    finally:
     if ssh:
      ssh.close()
    
    if output_lines:
    drupal_login_url = output_lines[0].replace("http://example.com/", "http://%s/" % settings.DRUPAL_SITE_URL).strip()
    
    destination = "%s?destination=%s" % (drupal_login_url, settings.DRUPAL_LOGIN_DESTINATION)
     return redirect(destination)
    else:
     return HttpResponse('<h1>Wrong request</h1>')
    

    This is the same code of the previous howto, with the difference that drush now is running on a different server of django. You can use the same method to do anything you have to with drush, any time you call this piece of code an SSH connection is opened.

    See also:

Create nice unicode PDF using Python

Today I started one of the less motivating activities in Python 2.x: encoding.

In Python 3 unicode will be everywhere, but as of the 2.6 version I’ve on one of the server I have to endure.

Objective: get data from a UTF-8 encoded json and print a nice PDF.

Tools: json, urllib2, fpdf, cgi

What you need:
pyfpdf: https://code.google.com/p/pyfpdf/downloads/list

  • Download fpdf-1.7.hg.zip or more recent
  • Unzip, enter the directory and python setup.py install
  • locate fpdf
  • cd /usr/lib/python2.6/site-packages/fpdf (or the directory name you got with locate)
  • Download unicode fonts for fpdf
  • Unzip and copy the fonts folder in the fpdf directory

Now you have a working FPDF with unicode support and unicode fonts. Start to write your script, I assume you’re using python 2.6, if not change python2.6 to your python version (e.g. 2.7) or remove version number in the heading (just python). As now FPDF works with Python 2.5 to 2.7.

Here I write a simple cgi-bin script, so you have to put it in the /var/www/cgi-bin directory (CentOS) or in /usr/lib/cgi-bin (Debian).

#!/usr/bin/env python2.6
#-*- coding: utf-8 -*-
from fpdf import FPDF
import json
import urllib2
import os
import cgi
import sys
# set system encoding to unicode
import sys
reload(sys)
sys.setdefaultencoding("utf-8")

Now get some arguments from url. These will be used to compile a query to a external json service.

# e.g. http://example.com/cgi-bin/myscript.py?lang=en&sid=2
sid = arguments.getlist('sid')[0]
lang = arguments.getlist('lang')[0]
# compile a request to get a particular element from an external json
dataurl = "http://example.com/external-json-source?lang=%s&sid=%s" % (lang, sid)
# load json from dataurl and convert into python elements
data = json.load(urllib2.urlopen(dataurl))
# the json has a user attribute: the user attribute has name and surname attributes as strings
user = data['user']
# title is a simple string
title = data['title']

Now you have to load the json from the external source. Json must be encoded in UTF-8:

lato_lungo = 297
lato_corto = 210
pdf = FPDF('L','mm','A4')
# add unicode font
pdf.add_font('DejaVu','','DejaVuSansCondensed.ttf',uni=True)
pdf.add_page()
pdf.cell(w=lato_lungo,h=9,txt=title,border=0,ln=1,align='L',fill=0)
pdf.set_font('DejaVu','',12)
# paragraphs rendered as MultiCell
# @see https://code.google.com/p/pyfpdf/wiki/MultiCell
# print key: values for each user['data'] dictionary attributes
for val in user.iteritems():
    pdf.multi_cell(w=0,h=5,txt="%s: %s" % val)
# finally print pdf
print pdf.output(dest='S')

Now:

  1. Open your browser and visit http://example.com/cgi-bin/myscript.py?lang=en&sid=2
  2. The external source http://example.com/external-json-source?lang=en&sid=2 is grabbed and converted into a python data structure. Both source and destination encoding are unicode utf-8.
  3. Data from external source are used to create the pdf.

You can use as many fonts as you have in the fpdf/font directory, just add those using pdf.add_font().

https://code.google.com/p/pyfpdf/downloads/list

Apache CentOS 6 cannot send email and Drupal get HTTP request status fails

I’m installing a Pressflow 6 on a new machine running CentOS 6. I’m using Apache MPM Worker with FastCGI. Then I get the classical e-mail error:

Unable to send e-mail. Please contact the site administrator if the problem persists.

Then I try to use sendmail:

sendmail -v yourmail@example.com < testmail

Where testmail is a file containing these lines:

Subject: test mail Ozu
Yasujiro Ozu
[blank line here]

And i get the message. PHP cannot send email through apache!

Trying a simple php script to send mail like drupal core do I got this error:

sendmail: fatal: chdir /var/spool/postfix: Permission denied

Then I check this variable following this awesome post:

# /usr/sbin/getsebool httpd_can_sendmail
httpd_can_sendmail --> off

Enable httpd_can_sendmail solve this issue:

setsebool -P httpd_can_sendmail 1

And wait. It will be a long wait using the -P option. And then PHP and Drupal can send mail.

Then check again the variable:

# /usr/sbin/getsebool httpd_can_sendmail
httpd_can_sendmail --> on

Now httpd can send mail. Try your script again.

The SMTP Authentication Support module is not working. This is another of these variables, the same that causes Drupal to show the “HTTP request status fails” message.

setsebool -P httpd_can_network_connect 1

And wait again. Both the SMTP module and the base Drupal networking are now working and Status report is all green.

Listen to your music and play video from your RaspberryPi NAS using DLNA

Note: DLNA will give access to your files on RaspberryPi through your local wireless network. Be careful choosing directories to expose.

Supposing you’ve already a RaspberryPi NAS:

sudo apt get install minidlna
sudo nano /etc/minidlna.conf

Change media dir to:

media_dir=A,/media/MYDRIVE/music/

where /media/MYDRIVE is the mount point and music is your music directory and A is the flag for Audio (leave unchanged for music, for pictures use P, for video use V).

Change db_dir (preview, database and cache directory) in:

db_dir=/media/MYDRIVE/cache/minidlna

Look for “friendly_name” and change it into something like:

friendly_name=RaspberryPiMusic

Then Ctrl+O to save.

Create the cache directory (as pi user, not superuser):

mkdir /media/MYDRIVE/cache
mkdir /media/MYDRIVE/cache/minidlna

Then use pi user for minidlna service (read previous howto for details about permissions and external drives).

sudo nano /etc/init.d/minidlna

And add USER=pi under the DEFAULT line:

PIDFILE=$PIDDIR/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
DEFAULT=/etc/default/$NAME
USER=pi

Then force reload the service, regenerating the cache and db:

sudo service minidlna force-reload

Now indexing is in progress: if you use a DLNA enabled device, like an Android phone (e.g. via DLNA application) or a Samsung Internet TV, you’ll got the list of files growing in number under the rasperrypi:RaspberryPiMusic server.

If you want to add new media directories, you have to add another media_dir to the list, specifying the media flag. I want to add my anime folder:

And then my anime folder:

media_dir=A,/media/MYDRIVE/music/
media_dir=/media/MYDRIVE/anime/

This time I omit the flag to play all contents, there are some mp3 too there and I want to play those!

Every time you add a new directory to watch, rebuild the database with force-reload, but if you want only to restart service use stop, start and restart instead of force-reload.

For a more detailed howto and the use of BubbleUPnP (shared playlist among devices) read this blog post by Stephen C Phillips, source of many info here.

Note: this post originated from this question by Fanie.

Follow

Get every new post delivered to your Inbox.

Join 26 other followers

%d bloggers like this: