As a python programmer, i feel it necessary to mention about the library search path in python. Python looks in several places in your system when we try and import a module in python.
Precisely , it searches in all directories mentioned in sys.path. Just try and see the directories in sys.path.
>>>import sys
>>>sys.path
This returns you a list of the directories where python does a 'look in' to import your module. This list may vary from system to system and depends on the python version that you use.For me, sys.path is :
['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/usr/lib/python2.5/site-packages/gst-0.10', '/var/lib/python-support/python2.5', '/usr/lib/python2.5/site-packages/gtk-2.0', '/var/lib/python-support/python2.5/gtk-2.0']
A few points about sys.path:
Importing the sys module makes all the attributes and functions of the module available to you.
sys.path as i mentioned gives you the current search path.
You could add a new directory to the python path by appending the list 'sys.path'
sys.path.append(/my_directory/my_setup)
Again view sys.path and you will find the directory mentioned above in the new search path.
Sunday, 5 September 2010
Functions in Python.
Just like most other languages , Python also has functions in it. Lets go through some of the aspects of functions in python.
def my_function(var1,var):
This is how you declare a function in Python.
The keyword def, the function name ,the arguments separeted by commas in parenthesis. Its important to understand that functions in python specify no return type. Each and every function returns something. Either the value returned or else 'none' , the python 'null value'. The arguments dont specify a type either , the type is made out from the values that are passed in .
Python is a :
dynamically typed language: The types are discovered at run time.
strongly typed language: One particular type cannot converted to another type without explicit type conversion.
Higher order functions in python: Functions that operate on other functions, (functions that can take other functions as its arguments).
Eg: filter(my_function,list_1) is a built in higher order function , which takes the function 'my_function' as one of its arguments.
Functions in python are first class:
Consider :
def my_function2(var1):
We have defined a function 'my_function2'
Now we could do this:
var=my_function.
Now try printing the value of 'var'
Its the address of the function 'my_function2' that will be printed because assigning a function name to a variable assigns it address to the variable.
Now we could invoke the function as:
var(my_parameter)
Just check out the glob module in python (/usr/lib/python2.5/glob.py) for a good example of the first class nature of functions in python.
Function composition is perhaps the best way to explain the fact that function are first class in python.
def f(my_funct):
def g(x):
def compose (f(g(x))): ----- function composition....functions are passed in as variables to other functions.
The way variables are used in function in also important . A good knowledge of the use of local and global variables are required to avoid any kind of misperception while using variables in python.
def my_function(var1,var):
This is how you declare a function in Python.
The keyword def, the function name ,the arguments separeted by commas in parenthesis. Its important to understand that functions in python specify no return type. Each and every function returns something. Either the value returned or else 'none' , the python 'null value'. The arguments dont specify a type either , the type is made out from the values that are passed in .
Python is a :
dynamically typed language: The types are discovered at run time.
strongly typed language: One particular type cannot converted to another type without explicit type conversion.
Higher order functions in python: Functions that operate on other functions, (functions that can take other functions as its arguments).
Eg: filter(my_function,list_1) is a built in higher order function , which takes the function 'my_function' as one of its arguments.
Functions in python are first class:
Consider :
def my_function2(var1):
We have defined a function 'my_function2'
Now we could do this:
var=my_function.
Now try printing the value of 'var'
Its the address of the function 'my_function2' that will be printed because assigning a function name to a variable assigns it address to the variable.
Now we could invoke the function as:
var(my_parameter)
Just check out the glob module in python (/usr/lib/python2.5/glob.py) for a good example of the first class nature of functions in python.
Function composition is perhaps the best way to explain the fact that function are first class in python.
def f(my_funct):
def g(x):
def compose (f(g(x))): ----- function composition....functions are passed in as variables to other functions.
The way variables are used in function in also important . A good knowledge of the use of local and global variables are required to avoid any kind of misperception while using variables in python.
Local and Global variables in python..
Lets see the working of global and local variables in python with the help of a few small examples: What you need to understand is that variables that are declared inside functions are local to that functions (cannot be accessed from outside the function), whereas variables that are declared outside the function are global to that function , they can be accessed within the function.
>>>a=1
>>>def func():
... print a
... a=10
...
>>>func()
would produce an error as:
UnboundLocalError: local variable 'a' referenced before assignment.
This is because the local variable 'a' is used in the 'print' statement before it is being assigned a value.
However ,
>>>a=1
>>>def func():
... global a
... print a
... a=10
...
>>>func()
wouldn't print an error of that sort.
Note the use of the keyword ' global' . This keyword indicates that the variable 'a' that is being printed is a global variable. The python interpreter understands that there is a variable 'a' that has been assigned to '1' outside the function and hence prints the value '1'.
Now do the following:
>>>a=1
>>>def func():
... global a
... print a
... a=10
... print a
>>>func()
The result would be:
1
10
Now try and print the value of 'a'.
The value printed would be:
'10'
This is because the value of the variable 'a' has been changed to 10 by the function
'func' because 'a' has been declared as a global variable in the function 'func'.
Now do :
>>>def func():
... print b
... b=10
...
>>>print b
would result in an error :
NameError: name 'b' is not defined
because no variable 'b' has been assigned a value outside the function. The variable 'b' used in the function 'func' is local to that function and cannot be accessed from that outside the function, that is , the variable 'b' is local to the function.
>>>a=1
>>>def func():
... print a
... a=10
...
>>>func()
would produce an error as:
UnboundLocalError: local variable 'a' referenced before assignment.
This is because the local variable 'a' is used in the 'print' statement before it is being assigned a value.
However ,
>>>a=1
>>>def func():
... global a
... print a
... a=10
...
>>>func()
wouldn't print an error of that sort.
Note the use of the keyword ' global' . This keyword indicates that the variable 'a' that is being printed is a global variable. The python interpreter understands that there is a variable 'a' that has been assigned to '1' outside the function and hence prints the value '1'.
Now do the following:
>>>a=1
>>>def func():
... global a
... print a
... a=10
... print a
>>>func()
The result would be:
1
10
Now try and print the value of 'a'.
The value printed would be:
'10'
This is because the value of the variable 'a' has been changed to 10 by the function
'func' because 'a' has been declared as a global variable in the function 'func'.
Now do :
>>>def func():
... print b
... b=10
...
>>>print b
would result in an error :
NameError: name 'b' is not defined
because no variable 'b' has been assigned a value outside the function. The variable 'b' used in the function 'func' is local to that function and cannot be accessed from that outside the function, that is , the variable 'b' is local to the function.
Version control Systems...
Version control is a system that records changes to your files and allows to view specific versions of a file or set of files.VCS is used so that you don't screw up your files or your entire project as a result of the silly , careless mistakes that you have made.
Many people do their own version control by copying their files to another directory.This is a quite simple and easy approach if you are careful enough so that you won't agonize by doing something disastrous so that you may lose all your stuff later.Therefore keeping in mind that such things can happen ,after all we are all humans, we use version control systems.
Centralized VCS and Distributed VCS.
Centralized VCS
CVCS includes a single server that contains different versions of the files and the clients could check into that.The major advantage of this system is that administrators have fine grained control over whats happening ,but it has serious drawbacks as well. They are:
Single point of storage if crash would be fatal.
The clients could pull anything from the repository but they need a commit access to push something into it.This might lead to a client getting frustrated if he/she has not been granted a 'commit access' even after working for a longer period of time. Also there might something political issues related to administration.
Distributed VCS.
In DVCS ,you could push data into the repository with the same ease with which you could pull from it. All the repositories are at the same level and therefore we could collaborate with any of the repositories.Examples of DVCS are Git , mercurial , bazaar..etc...
Many people do their own version control by copying their files to another directory.This is a quite simple and easy approach if you are careful enough so that you won't agonize by doing something disastrous so that you may lose all your stuff later.Therefore keeping in mind that such things can happen ,after all we are all humans, we use version control systems.
Centralized VCS and Distributed VCS.
Centralized VCS
CVCS includes a single server that contains different versions of the files and the clients could check into that.The major advantage of this system is that administrators have fine grained control over whats happening ,but it has serious drawbacks as well. They are:
Single point of storage if crash would be fatal.
The clients could pull anything from the repository but they need a commit access to push something into it.This might lead to a client getting frustrated if he/she has not been granted a 'commit access' even after working for a longer period of time. Also there might something political issues related to administration.
Distributed VCS.
In DVCS ,you could push data into the repository with the same ease with which you could pull from it. All the repositories are at the same level and therefore we could collaborate with any of the repositories.Examples of DVCS are Git , mercurial , bazaar..etc...
Friday, 3 September 2010
Shebang lines...
Shebang(also known as 'hashbang') refers to the characters #! when they are the first two characters (occurring at the left most corner, i.e the beginning of the very first line) of your script. The particular line that contains the shebang is the shebang line in your script.
Now what's shebang meant to do.?
In a unix-like operating system , the program loader (part of the operating system thats responsible for loading programs) finds the presence of these two characters(#!) , identifies it as a script and tries to execute it with the interpreter that will be specified using the remaining lines of the script.
For example consider writing a shebang line for a python script..
First and foremost you need to find the location of the interpreter for that particular script...
$which python
/usr/bin/python
Therefore...the shebang line should be set as...
#! /usr/bin/python
Specifying the shebang line would execute your script as follows:
./'script name'.py instead of doing 'python 'script name'.py'
You could try out the use of shebang lines for any of the following types of scripts
'werbel', 'tinypy' ,'perl'.....and many more...
Now what's shebang meant to do.?
In a unix-like operating system , the program loader (part of the operating system thats responsible for loading programs) finds the presence of these two characters(#!) , identifies it as a script and tries to execute it with the interpreter that will be specified using the remaining lines of the script.
For example consider writing a shebang line for a python script..
First and foremost you need to find the location of the interpreter for that particular script...
$which python
/usr/bin/python
Therefore...the shebang line should be set as...
#! /usr/bin/python
Specifying the shebang line would execute your script as follows:
./'script name'.py instead of doing 'python 'script name'.py'
You could try out the use of shebang lines for any of the following types of scripts
'werbel', 'tinypy' ,'perl'.....and many more...
Wednesday, 1 September 2010
Optimizing your Python code.
Compare python with some of the other languages that are familiar to many of you. The obvious difference that will be caught by you , is the fact that python code, for the same program ,is much shorter than codes written in some of your favorite languages like C ,Java etc.
Understanding the essence of python programming would help you write shorter and interesting codes and are easier to understand. Lets just have a look into some of the core points that need to be followed so that your python program becomes short and effective.
First and foremost, you need to understand the fact that python has got a huge library with functions for almost every activity you would need frequently. The only thing you need to do is to identify whether a function is present that would handle your case and then call the function .There are lots of built in functions that we could call. Other functions that reside in certain modules can also be invoked after the module containing the function has been imported.
As an example , consider the following:
map(sqr,range(1,100)):
'map' is a built -in higher order function that takes two or more arguments and returns you a list,'range' is another built-in which would return a list containing all the elements from the starting element up to the ending element.
This particular function returns a list of squares of all numbers from 1 up to 99 using a single line code. Compare this to a C code to find the squares of this many numbers,certainly ,a single line code wouldn't do that for you.
Python has got a large collection of such built ins which includes functions like 'filter','reduce','hex','cmp' and many more.
Python has got many more functions that are defined in different modules:To use those functions , you just need to import those modules and invoke them. Some of the most commonly invoked ones are :
Module 're' for pattern matching. Contains power full functions like findall' , search' , 'match' etc.
Module 'os' for operating system services. Includes functions 'mkdir' , 'chdir' ,' path.join,' path.abspath'' etc.
Module 'shutil' for shell utilities like copying files.
Module 'urllib' for dealing with 'URLs'.
Module 'sys' :sys.argv[index] would return the input given on the command line at position 'index.'
Identifying the correct functions that are present in these and more modules in python would result in much shorter code length.
A proper knowledge about the use of sequences and variables would do no harm in our purpose of decreasing the code size.
Consider the code to swap the values of two variables. In other languages you would write:
temp=a:
a=b:
b=temp:
Once again, in python all you need is a single line code :
a,b=b,a
The logic behind whats happening is to be understood. Consider the right hand side of the expression -variables are packed into a tuple. On the left hand side, the tuple is unpacked and the values are entered into the variables. Thus the values are swapped in a single line as opposed to three lines taken in other languages.
Use of generator expressions is another way to reduce your code size.
Consider:
xvec=[10,20,30]
yvec=[30,40,50]
The expression sum(x*y for x,y in zip(xvec,yvec)) gives you the dot product of xvec and yvec in a single step where 'sum' is another built in function. Think of writing a C code instead of this and the number of writing you would take.
There are many more such tricks that would decrease the code size considerably . Slicing of lists , usage of the appropriate sequences are all important in producing a short and elegant code. Obviously,the fact remains that to understand the methodology behind decreasing the code size is to keep on coding ,understanding the definitions of the functions and identifying where to use them. But again, compressing your code size heavily might result in your code being hard to digest by others, specially those beginners trying to understand a python code. Therefore its important to develop a skill where you would maintain a balance between decreasing the code size and the readability of your code. This can be achieved only through rigorous coding.
Understanding the essence of python programming would help you write shorter and interesting codes and are easier to understand. Lets just have a look into some of the core points that need to be followed so that your python program becomes short and effective.
First and foremost, you need to understand the fact that python has got a huge library with functions for almost every activity you would need frequently. The only thing you need to do is to identify whether a function is present that would handle your case and then call the function .There are lots of built in functions that we could call. Other functions that reside in certain modules can also be invoked after the module containing the function has been imported.
As an example , consider the following:
map(sqr,range(1,100)):
'map' is a built -in higher order function that takes two or more arguments and returns you a list,'range' is another built-in which would return a list containing all the elements from the starting element up to the ending element.
This particular function returns a list of squares of all numbers from 1 up to 99 using a single line code. Compare this to a C code to find the squares of this many numbers,certainly ,a single line code wouldn't do that for you.
Python has got a large collection of such built ins which includes functions like 'filter','reduce','hex','cmp' and many more.
Python has got many more functions that are defined in different modules:To use those functions , you just need to import those modules and invoke them. Some of the most commonly invoked ones are :
Module 're' for pattern matching. Contains power full functions like findall' , search' , 'match' etc.
Module 'os' for operating system services. Includes functions 'mkdir' , 'chdir' ,' path.join,' path.abspath'' etc.
Module 'shutil' for shell utilities like copying files.
Module 'urllib' for dealing with 'URLs'.
Module 'sys' :sys.argv[index] would return the input given on the command line at position 'index.'
Identifying the correct functions that are present in these and more modules in python would result in much shorter code length.
A proper knowledge about the use of sequences and variables would do no harm in our purpose of decreasing the code size.
Consider the code to swap the values of two variables. In other languages you would write:
temp=a:
a=b:
b=temp:
Once again, in python all you need is a single line code :
a,b=b,a
The logic behind whats happening is to be understood. Consider the right hand side of the expression -variables are packed into a tuple. On the left hand side, the tuple is unpacked and the values are entered into the variables. Thus the values are swapped in a single line as opposed to three lines taken in other languages.
Use of generator expressions is another way to reduce your code size.
Consider:
xvec=[10,20,30]
yvec=[30,40,50]
The expression sum(x*y for x,y in zip(xvec,yvec)) gives you the dot product of xvec and yvec in a single step where 'sum' is another built in function. Think of writing a C code instead of this and the number of writing you would take.
There are many more such tricks that would decrease the code size considerably . Slicing of lists , usage of the appropriate sequences are all important in producing a short and elegant code. Obviously,the fact remains that to understand the methodology behind decreasing the code size is to keep on coding ,understanding the definitions of the functions and identifying where to use them. But again, compressing your code size heavily might result in your code being hard to digest by others, specially those beginners trying to understand a python code. Therefore its important to develop a skill where you would maintain a balance between decreasing the code size and the readability of your code. This can be achieved only through rigorous coding.
Garbage collection in Python.
In C, once the memory allocated becomes garbage , that chunk of memory is gone wasted. Nothing more is possible on it.
Consider
int a[10];
Some amount of memory would be allocated for the array 'a'. Now if do a='some value' , the memory that was allocated to the array becomes garbage because 'a' no longer uses it as it is pointing to something else now. The memory is gone wasted and is no more usable.
The bugs that create loss of memory are hard to detect as well ,simply because of the fact that they cannot be seen. Its only after executing the particular code for some amount of times that you might actually get to see the effects of the bug .By the time your system may have crashed.
In python ,the interpreter is smart enough to do 'garbage collection'. That is the memory which is unused is collected by the interpreter and is not gone wasted. The process of garbage collection is done by using a method called ref counting. Understand ref counting as :
Consider a=[1,2,3]
'a' points to a particular block of memory. At the end of the block of memory ,there would be a count that indicates the number of variables that point to the same block of memory. So now the count would be 1.
Now if we do,
b=a......means that now b points to same block of memory. Therefore the count becomes 2.
Now consider doing
b=0.
The count in the block of memory where b was pointing earlier reduces by 1. , hence count is now 1.
Do
'a=0'
and the count in that block becomes 0.Now the python interpreter can allot the chunk of memory to some other variable as it may feel. The significant thing that you need to understand is the fact that the memory is not gone wasted as in C .
Garbage collection keeps track of the variables that point to a block of memory and collects the unused memory accordingly. There is never a shortage of memory space. But there are obvious disadvantages with the process of garbage collection.
One drawback , as you would except would be the matter of speed which would decrease.
But the more significant drawback would be the non-deterministic behavior of the interpreter.
Consider the two statements.
.i=0
.j=0
You would expect the statement j=0 to be executed immediately after the statement i=0 has been executed. But as a result of this process of garbage collection ,this is not a guarantee. There might be a certain time delay in executing the second statement if the interpreter feels like doing some 'garbage collection' after the first statement has been executed . This results in the delay.
Therefore its clear that its not best to use python over languages like C when it comes to real time systems.
Consider
int a[10];
Some amount of memory would be allocated for the array 'a'. Now if do a='some value' , the memory that was allocated to the array becomes garbage because 'a' no longer uses it as it is pointing to something else now. The memory is gone wasted and is no more usable.
The bugs that create loss of memory are hard to detect as well ,simply because of the fact that they cannot be seen. Its only after executing the particular code for some amount of times that you might actually get to see the effects of the bug .By the time your system may have crashed.
In python ,the interpreter is smart enough to do 'garbage collection'. That is the memory which is unused is collected by the interpreter and is not gone wasted. The process of garbage collection is done by using a method called ref counting. Understand ref counting as :
Consider a=[1,2,3]
'a' points to a particular block of memory. At the end of the block of memory ,there would be a count that indicates the number of variables that point to the same block of memory. So now the count would be 1.
Now if we do,
b=a......means that now b points to same block of memory. Therefore the count becomes 2.
Now consider doing
b=0.
The count in the block of memory where b was pointing earlier reduces by 1. , hence count is now 1.
Do
'a=0'
and the count in that block becomes 0.Now the python interpreter can allot the chunk of memory to some other variable as it may feel. The significant thing that you need to understand is the fact that the memory is not gone wasted as in C .
Garbage collection keeps track of the variables that point to a block of memory and collects the unused memory accordingly. There is never a shortage of memory space. But there are obvious disadvantages with the process of garbage collection.
One drawback , as you would except would be the matter of speed which would decrease.
But the more significant drawback would be the non-deterministic behavior of the interpreter.
Consider the two statements.
.i=0
.j=0
You would expect the statement j=0 to be executed immediately after the statement i=0 has been executed. But as a result of this process of garbage collection ,this is not a guarantee. There might be a certain time delay in executing the second statement if the interpreter feels like doing some 'garbage collection' after the first statement has been executed . This results in the delay.
Therefore its clear that its not best to use python over languages like C when it comes to real time systems.
Subscribe to:
Posts (Atom)