Wednesday, December 23, 2015

Django :: IOError: encoder error -2 when writing image file


exceptions:IOError: encoder error -2 when writing image file

Traceback (most recent call last):
 File "/django_virtalenv//lib/python2.7/site-packages/newrelic-2.14.0.11/newrelic/hooks/framework_django.py", line 492, in wrapper return wrapped(*args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/contrib/admin/options.py", line 465, in wrapper return self.admin_site.admin_view(view)(*args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view response = view_func(request, *args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func response = view_func(request, *args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/contrib/admin/sites.py", line 198, in inner return view(request, *args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper return bound_func(*args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view response = view_func(request, *args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func return func(self, *args2, **kwargs2)
 File "/django_virtalenv//lib/python2.7/site-packages/django/db/transaction.py", line 371, in inner return func(*args, **kwargs)
 File "/django_virtalenv//lib/python2.7/site-packages/django/contrib/admin/options.py", line 1263, in change_view self.save_model(request, new_object, form, True)
 File "/data/cms/trunk/cms/articles/admin.py", line 125, in save_model obj.save()
 File "/django_virtalenv//lib/python2.7/site-packages/django/db/models/base.py", line 545, in save force_update=force_update, update_fields=update_fields)
 File "/django_virtalenv//lib/python2.7/site-packages/django/db/models/base.py", line 573, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
 File "/django_virtalenv//lib/python2.7/site-packages/django/db/models/base.py", line 632, in _save_table for f in non_pks]
 File "/django_virtalenv//lib/python2.7/site-packages/django/db/models/fields/files.py", line 252, in pre_save file.save(file.name, file, save=False)
 File "/data/cms/trunk/cms/utils/thumbs.py", line 93, in save thumb_content = generate_thumb( content, size, split[1] )
 File "/data/cms/trunk/cms/utils/thumbs.py", line 60, in generate_thumb image2.save( io, "JPEG", quality = 90, optimize=True, progressive=True)
 File "/django_virtalenv//lib/python2.7/site-packages/PIL/Image.py", line 1439, in save save_handler(self, fp, filename)
 File "/django_virtalenv//lib/python2.7/site-packages/PIL/JpegImagePlugin.py", line 471, in _save Image
 File._save(im, fp, [("jpeg", (0,0)+im.size, 0, rawmode)])
 File "/django_virtalenv//lib/python2.7/site-packages/PIL/Image
 File.py", line 491, in _save raise IOError("encoder error %d when writing image file" % s)IOError: encoder error -2 when writing image file

Soln :- pip install Pillow

Thursday, December 17, 2015

Performance Guidelines for making webpages load more efficiently / Page Speed

Sno Activity Definition
1 Minimize HTTP Requests Each image, script file, css and other external files are counted as 1 HTTP request individually
2 Use a Content Delivery Network CDNs allow faster downloading of files at Users end, as they have tie ups with ISPs directly.
Similar to LAN transfer. Eg. Akamai
3 Avoid empty src or href IE - Makes a request to directory in which the page is located
Chrome/Safari/Firefox - Make request to original page
This can create Errors and useless Extra Traffic at Web Server
4 Add an Expires or a Cache-Control Header For static components: implement "Never expire" policy by setting far future Expires header
For dynamic components: use an appropriate Cache-Control header to help the browser with conditional requests
5 Gzip Components Gzip is a compression algorithum which can reduce ~70% file size of static/text based content.

Smaller file size, faster loading and lesser bandwidth consumed.
6 Put StyleSheets at the Top Stylesheets (CSS) files by nature can be parallely loaded along with other static content like images.
Loading them early also helps in faster rendering of page.
7 Put Scripts at the Bottom Browsers open 2 connections per Hostname (aka Parallel Downloading.)
While a script is downloading the browser won't start any other downloads, even on different hostnames.
8 Avoid CSS Expressions Used to set CSS properties dynamically. Especially for IE. Powerful and dangerous. As they are recalculated on every mouse movement and keyboard input.
9 Make JavaScript and CSS External Reduces size of main HTML page at the cost of extra HTTP requests. However, if CSS/JS are cached properly then HTTP requests can also be saved.
10 Reduce DNS Lookups DNS maps hostnames to IP addresses, just as phonebooks map people's names to their phone numbers. Takes 20-120 mil secs to resolve a hostname to IP. Browser cannot download anything uptill the DNS is not resolved.
11 Minify JavaScript and CSS Minification is the practice of removing unnecessary characters from code to reduce its size thereby improving load times.
12 Avoid Redirects Redirects slow down the user experience.
Inserting a redirect between the user and the HTML document delays everything in the page since nothing in the page can be rendered and no components can start being downloaded until the HTML document has arrived.
13 Remove Duplicate Scripts If a JS file is referred to twice for same page:
IE - Will download it again, even if JS is cached
However, all browsers even if they don't download it, will always execute it twice, that is unwated time spent on processing.
14 Configure ETags Entity tags (ETags) are a mechanism that web servers and browsers use to determine whether the component in the browser's cache matches the one on the origin server.
Can be a boon or a curse.
Single web server - Definitely a boon
Multiple web servers - Needs evaluation, can even degrade performance
15 Make AJAX Cacheable Ajax responses should be made cacheable, this will significantly improve User Experience
16 Use GET for AJAX Requests Ajax requests (XMLHttpRequest)
POST - Sends headers then sends Data.
GET - Sends Data directly. However, cap on URL length is 2,000 chars, data larger than that should be sent using POST method.
17 Reduce the Number of DOM Elements A complex page design means more bytes to download and it also means slower DOM access in JavaScript.
It makes a difference if you loop through 500 or 5000 DOM (design/HTML) elements on the page when you want to add an event handler for example.
To get count of DOM elements FireBug > Console >
document.getElementsByTagName('*').length
Ideal size - 500 to 700 elements for a very busy page
18 No 404s HTTP requests are costly and getting useless response will further bring down user experience.
1) Some sites keep helpful 404 pages, however this can lead to unwanted DB hits eg. "Did you mean X".
2) When external JS files return 404.
a) First it blocks browser from loading anything else,
b) Secondly the browser might try to parse the error page to find something useable.
19 Reduce Cookie Size Bloating cookies with unwanted information can also slow down user responses. As cookies travel with every request user makes.
Cookie size - <400 Bytes for any single cookie
20 Use Cookie-Free Domains for Components As cookies travel with each request, static content (images/css/etc) that don't require them should be served from cookie free domains as this will speed up trasmission.
21 Avoid Filters The IE-proprietary AlphaImageLoader filter aims to fix a problem with semi-transparent true color PNGs in IE versions < 7.
Blocks rendering
Freezes the browser
Increases memory comsumption
Applied per element, not per image, so the problem is multiplied.
Solution -
1) Use gracefully degrading PNG8 instead
2) If you absolutely need AlphaImageLoader, use the underscore hack _filter as to not penalize your IE7+ users.
22 Do Not Scale Images in HTML Optimize image resolution to match front end requirement don't scale down a large size image in HTML.
23 Make favicon.ico Small and Cacheable It's a necessary evil and Interferes with the download sequence. To set it right
Size - <1 Kb
Cache - Long term
Serving - Cookie-less domain
24 Avoid bad requests Same as No 404, bad requests are generated by User (4xx) and Server (5xx) - So all 4xx and 5xx series errors should be addressed
25 Avoid CSS @import Changes default nature of CSS download from Parallel to Serial. This is similar to loading CSS at the end of the page, which is a bad practice.
26 Avoid CSS expressions As mentioned earlier
27 Avoid document.write If external resources (JS/CSS/Images/etc) are invoked via document.write they cannot be prefetched.
28 Combine external CSS Club all CSS into 1 file
29 Combine external JavaScript Club all JS into 1
30 Combine images using CSS sprites Each request incurs a fixed amount of request overhead. Reduce this overhead from one request per image to one request for the entire sprite.
31 Defer loading of JavaScript Identify JS functions not used by document before onload event. Load them seperately using event handler.

Alternately "defer" can be used as a mode in HTML4/5
<script type="text/javascript" defer="defer">
alert(document.getElementById("p1").firstChild.nodeValue);
</script>
32 Defer parsing of JavaScript
33 Enable compression Same as Gzip components
34 Leverage browser caching Setting an Expiry/Max-age header for a resource helps browsers to cache items locally in user machines
35 Leverage proxy caching Enabling public caching in the HTTP headers for static resources allows the browser to download resources from a nearby proxy server rather than from a remote origin server.
This means that even first-time users to your site can benefit from caching. Similar to CDN.
36 Make landing page redirects cacheable Redirects slow down the user experience. If they are not cached to do so:
Expires or Cache-Control header must be added to response
Mobile phone redirects should be handled seperately
37 Minify CSS Similar to Minify JavaScript and CSS
38 Minify HTML Compacting HTML code, including any inline JavaScript and CSS contained in it, can save many bytes of data and speed up downloading, parsing, and execution time.
39 Minify JavaScript Similar to Minify JavaScript and CSS
40 Minimize request size HTTP request should not go beyond 1 packet. That is ~1500 bytes.
Cookie size - < 400 Bytes.
URL length - < 500 Bytes including parameters
Browser-fields - Request/Response headers, only set required ones
41 Minimize DNS lookups Similar to Reduce DNS lookups
42 Minimize redirects Similar to Avoid redirects
43 Optimize images Choose an appropriate image file format
PNGs are almost always superior to GIFs
GIFs to be used for very small or simple graphics
JPG for photographic style images
BMP and Tiff to be avoided
Use an image compressor
JPG - JPEGtran or JPEGoptim
PNG - OptiPNG or PNGout
44 Optimize the order of styles and scripts <head>
<!-- Title -->
<!-- Meta -->
<!-- CSS -->
<!-- Favicon -->
<!-- JS if required (inline/external) -->
</head>
45 Parallelize downloads across hostnames 100 resources
4 hosts
each host should serve 25 resources
no one host should serve more than 38 resources

*** Also many browsers do not download JavaScript files in parallel, so there is no benefit from serving them from multiple hostnames
46 Prefer asynchronous resources Fetching resources asynchronously prevents those resources from blocking the page load.

Example
<script>
var node = document.createElement('script');
node.type = 'text/javascript';
node.async = true;
node.src = 'example.js';
// Now insert the node into the DOM, perhaps using insertBefore()
</script>
47 Put CSS in the document head Similar to Put stylesheets at the top
48 Remove unused CSS Removing or deferring style rules that are not used by a document avoid downloads unnecessary bytes and allow the browser to start rendering sooner
49 Serve resources from a consistent URL Shared resources across multiple pages in a site such as images, .css and .js files should always be served from a consistent URL. This greatly helps in reducing bandwidth comsumption for both User and Server.
50 Serve scaled images If a page contains a large image and thumbnail of the same is to displayed, the same large image can be scaled down and reused, provided the aspect ratio for thumbnail is maintained.
51 Serve static content from a cookieless domain Similar to Use Cookie-Free Domains for Components
52 Specify a character set Add Content-Type with charset response header
Content-Type: text/html; charset=utf-8
This brings extra buffering to the page, as the first thing a browser does is to try and obtain most suitable charset to display information on a page

NOTE : Both charset header and <meta> charset should be identical for a page
53 Specify image dimensions Specify height and width at <img> element level
This helps in reserving space for image before it loads and boosts page rendering
54 Use efficient CSS selectors 1) Avoid a universal key selector.
Allow elements to inherit from ancestors, or use a class to apply a style to multiple elements.

2) Make your rules as specific as possible.
Prefer class and ID selectors over tag selectors.

3) Remove redundant qualifiers.
These qualifiers are redundant:
a) ID selectors qualified by class and/or tag selectors
b) Class selectors qualified by tag selectors (when a class is only used for one tag, which is a good design practice anyway).

4) Avoid using descendant selectors, especially those that specify redundant ancestors.
For example, the rule body ul li a {...} specifies a redundant body selector, since all elements are descendants of the body tag.

5) Use class selectors instead of descendant selectors.

instead of using 2 style rules to display ordered list and ordered list
ul li {color: blue;}
ol li {color: red;}

encode the styles into two class names and use those in your rules
.unordered-list-item {color: blue;}
.ordered-list-item {color: red;}
55 Implementing mod pagespeed for apache Apache module to bring page speed recommendations aboard
56 PHP flush(); Putting PHP Flush after the </head> segment can improve performance significantly
... <!-- css, js -->
</head>
<?php flush(); ?>
<body>
... <!-- content -->
57 Implement Xcache / ZendOpcache The Xcache / Zend OPcache provides faster PHP execution through opcode caching and
optimization. It improves PHP performance by storing precompiled script
bytecode in the shared memory. This eliminates the stages of reading code from
the disk and compiling it on future access. In addition, it applies a few
bytecode optimization patterns that make code execution faster.
58 Post-load content What's absolutely required in order to render the page initially?
The rest of the content and components can wait.
YUI Image Loader - Loads images in the visible scroll
YUI GET Utility
59 Pre-load content Unconditional Preload - If you visit Google's homepage, it preloads all required files for Search Result page.
Conditional Preload - Yahoo search preloads extra components basis search terms
Anticipated Preload - When planning a redesign, preload all new design requisite on older version of pages
60 Minimize Iframes Costly even if blank
Affects initialization time of onload event
61 Optimize css sprites Smaller size - Arrange images horizontally instead of vertical
Optimize - Combining similar colors in a sprite helps you keep the color count low, ideally under 256 colors so to fit in a PNG8.
Mobile friendly - Don't leave big gaps between images
62 iPhone caching iPhone doesn't cache components larger than 25KB in size
63 Mobile phone redirects Mobile-specific redirects must be privately cacheable.
Else proxies will serve mobile redirect to non-mobile users. 1) Use a 302 redirect with a cache lifetime of one day.
2) To avoid mobile redirect to non-mobile user redirect should contain
a) Vary: User-Agent header
b) Cache-Control: private header.
c) Expires header in the past, to prevent old HTTP/1.0 proxies from caching these redirects
64 W3C Validation of CSS & HTML This speeds up page drawing and rendering in the browser as there is no extra computation done from browsers end to display the page properly
Ref :- http://yslow.org/ http://yslow.org/ruleset-matrix/ https://developers.google.com/speed/

Python :: Spammed word Validator or Blocked Spammed Word

Python :: Spammed word Validator or Blocked Spammed Word :-

def is_spammed(content):
    content = " "+content.lower()+" "
    spamword_list = ['call girls','Escort Service','sex','spam']
    for x in spamword_list:
        if " "+x+" " in content:
            return True
    return False

is_spammed("this is spam word") // return True

Wednesday, December 16, 2015

Django :: ...(remaining elements truncated)... Issue

Django :: ...(remaining elements truncated)... Issue :-

Problem :: ...(remaining elements truncated)... data was saved into mysql db ?

Why Occurs :: Because of List object was trying to saved into field of mysql db by python str() function due to which it truncate list and replace larger list data to ...(remaining elements truncated)...

Soln :- Replace str() to list()

Mysql :: How can I dump a specific table or set of tables without including the rest of the db tables?

Mysql :: How can I dump a specific table or set of tables without including the rest of the db tables?

Soln:-

mysql -u -p -e 'show tables like "_%"' | grep -v Tables_in | xargs mysqldump -u -p 

mysql dump data only :-
mysqldump --no-create-info

mysql dump structure only :-
mysqldump --no-data