From the Oxfam website:

Biofuels may offer the potential to reduce poverty by increasing jobs and markets for small farmers, and by providing cheap renewable energy for local use, but the huge plantations emerging to supply the EU pose more threats than opportunities for poor people. The problem will only get worse as the scramble to supply intensifies unless the EU introduces safeguards to protect land rights, livelihoods, workers rights and food security.

...

The UN estimates that 60 million people worldwide face clearance from their land to make way for biofuel plantations. Many end up in slums in search of work, others on the very plantations that have displaced them with poor pay, squalid conditions and no worker rights. Women workers are routinely discriminated against and often paid less then men.

In Indonesia almost a third of palm oil is produced by smallholders most of whom lost their land to advancing plantations and were ‘rewarded’ with a two hectare plot. These smallholders are bonded to the palm oil companies which provide them with credit and are required to sell to them – which means they do not get the best price for their oil.

We've got a team on the ground in Indonesia documenting the situation first hand, and joining locals in direct action. Read more in the Indonesian Forest Defenders Camp blog.

From Wired:

Jean Ziegler, the U.N.'s independent expert on the right to food, recently called for a five-year moratorium on biofuels, saying, "It is a crime against humanity to convert agricultural productive soil into soil which produces food stuff that will be burned into biofuel." He argued converting crops to fuel is driving up food costs and poor nations could become unable to feed their people.

Times of India:

Among the negative effects of fuel production from corn, sugarcane and other crops are the indiscriminate felling of forests, and high price and shortage of food, says the UN Energy report.

Many species of trees would disappear, so ecosystems that absorb carbon from the atmosphere would be destroyed, leading to an increase in polluting emissions, the reports points out.