Core prices, which exclude those of volatile fresh food, fell 0.1 percent in March from a year earlier, the first drop since September 2007, the government reported.
Japan was stuck in a deflationary spiral for years after its economic bubble burst in the early 1990s, eroding corporate earnings and encouraging consumers to delay their spending in the hope that prices would fall further.
Inflation hit a decade high 2.4 percent briefly last year but has since evaporated against a backdrop of falling energy costs and weak domestic demand, amid the country's worst recession in decades.
Japanese consumer spending fell for a 13th straight month in March, down 0.4 percent from a year earlier, the government said.
With Japan's benchmark interest rate already at 0.1 percent, the central bank has limited ammunition left to stop prices from falling.
Japan's central bank has warned that Asia's biggest economy faces two years of deflation, with consumer prices expected to drop 1.5 percent this year and 1.0 percent the following year.
Core consumer prices in Tokyo, a leading indicator released earlier than figures for the whole of Japan, were flat in April, the government said.
Japan's economy suffered a brutal annual contraction of 12.1 percent in the last three months of 2008, putting it on course for its worst recession since World War II.
Recent figures on exports and output have raised hopes that the slump may be easing, although prospects for a full recovery remain dim as long as the global economy remains weak, due to Japan's heavy dependence on foreign markets.