Forums / Developer / Import XML Data Topic

Import XML Data Topic

Author Message

Kristof Coomans

Friday 03 November 2006 5:22:55 am

If the parser does not return an object then you can get any parsing errors with

$errors = $parser->getMessages();

and print them

foreach ( $errors as $error )
{
    $cli->output( $error );
}

independent eZ Publish developer and service provider | http://blog.coomanskristof.be | http://ezpedia.org

Maurizio Betti

Sunday 19 November 2006 2:39:43 pm

Hi all,

I need to import xml data for two different classes with relations (storing datatype ezobject_relations). Is it possibile in some way? Any idea to do this?

Any suggestion will be very appreciated. Thanks in adavance.

Maurizio Betti

Maurizio Betti
http://www.arsweb.it
http://www.hospes.it

*- pike

Sunday 19 November 2006 5:02:07 pm

Hi Maurizio

my xmlimport extension should be able to do this. it isn't published in the contributions yet; i'm waiting for a response from olivier pierret. it's tested with the 'objectrelationbrowse' datatype, but 'objectrelation' and 'objectrelationlist' should work, too.

take a close look at these screensnaps:
http://pike.kw.nl/browse/files/projects/pike/2006/ezpublish/XMLImport/screensnaps
and notice the relations in the xmlfile.

as you see, relations are nested. inside the tag of the relationattribute, there are entries of the right datatype with a set of fields that should together form a unique identifier of the related resource. the importscript will perform a search on those. if it finds multiple applicable objects, it will return a warning and use the first one; if it doesn't find any applicable object, it will only issue a warning. see the output screensnaps for examples. ofcourse, the related object should have been imported before the relating object, it should already exist.

if you think you need this, contact me (pike-ez [AT] kw.nl)

*-pike

---------------
The class eZContentObjectTreeNode does.

Daniel Bins

Monday 20 November 2006 9:01:42 am

I have a question about import data using ImportXMLData contrib.

In my job, I used this contrib to import data, with sucess. But, after install this in my client (using the same version of EZ, 3.7 with brazilian portuguese as default language), I have problems with some caracters. For example, in the word "não" the "ã" is replaced with a square.

Using a MYSQL Front End, I look the tables where the data is imported (I dont remember what) and the text is stored right. The word "não" for example, is stored "não". In the source code of the browser (IE - View - Source Code), the words are OK, but, the navigator show this page with squares replaced the accents.

In first time, i suposed was problem with the encoding of the page (in IE, View - Encoding), but changes in this nothing solve the problem. I try changed the encoding of the XML file, without sucess.

Searching in the forum, I foung that one solution was edit the PHP class and add some changes.($parser = xml_parser_create('UTF-8');) In my job, works perfectly, in the client results in "page not found".

To understand better the problem, the client send me your database, to compare with the original database, I found that in my database, a word like "não" is stored exactly. No matter if the data is inserted with EZ or imported, the text is stored exactly as inserted, and I have no problem to view in browser.

In the client database, with exception of data imported with XML, the data inserted using EZ replaced the accents with some codification. To my client, view of this data are OK, but view data imported no.

What is the solution for my problem? Is possible to import data using other file format, like Excel or Access?

*- pike

Tuesday 21 November 2006 2:01:49 pm

Hi Maurizio

a complicated story, as usual with encodings :-)

in your office, mysql4, ezp3.7,
- stuff in the browser looks ok
- the mysql source seems (?) ok

at your clients office, mysql unknown, ezp3.7,
- there are strange characters in the browser
- the mysql source seems (?) ok

then you notice something else:

in your clients database,
- strange chars inserted via the admin get some encoding
- strange chars inserted via the import are stored literally

in your own database,
- strange chars inserted via the admin are stored literally
- strange chars inserted via the import are stored literally

is that all correct ?
so at both offices, working with the admin is no problem,
but the databasecontent is different, one 'literally', one
'with some encoding'.

i have no clue :-)) but consider where you want to go.
do you want to

- keep working at your clients office with that encoding, iow
- leave their setup (and admin) as it is
- find out what goes wrong with the import, or

- get everything working at the clients office like in your own office, iow
- compare their setup with yours, make it the same
- leave the importscript as it is, probably working by then

I would go for the 2nd option, compare the mysql version, ini files,
ezp settings, line everything up. there must be some difference.

*-pike

ps. did you add the <?xml header to your import file ?

---------------
The class eZContentObjectTreeNode does.

Daniel Bins

Monday 27 November 2006 6:38:30 pm

Thanks pike. You are right!

The problem was the encoding of XML file. In my office, with or without the encoding line in the XML file, the import are OK. In the other server, nothing. The XML file was created in Windows Notepad.

But, the client edit this XML file in a Linux machine, insert the encoding line, save the file and import again. And this time, the import data are OK. The same process in Windows notepad do not works.

Badii Bouteraa

Tuesday 20 February 2007 10:44:50 am

Hello,

I try to integrate ImportXMLData contrib in a multi-langague site and I want to know if somebody has an idea on positivity to add the option to select site language in the imported xml data.

Sorry for my English

mocifuen Cifu

Thursday 22 March 2007 6:28:46 am

Hi all;

I have installed the extantion and it works ok.
But it seem that I have memory problems... When trying to import files of ca. 60 KB (Not very big files) I receive this message:

<i>
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 2 bytes) in /var/www/no.hia.ezweb/ezpublish/base/var/au/cache/trans/g-lowercase-635518252-utf-8.ctt.php on line 274
Fatal error: eZ publish did not finish its request
</i>

If I repeat the same action with the same file, this message appears:
<i>
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 18000 bytes) in /var/www/no.hia.ezweb/ezpublish/base/lib/ezi18n/classes/ezchartransform.php on line 296
Fatal error: eZ publish did not finish its request
</i>

I added: <i> $attribute->setAttribute("data_text",utf8_encode($xml_content_string)) </i>
To the importXMLDatafunctioncollection.php file to import the norwegian letters into the database.

Any solutions?
Commandline? or other fixes.

Thanx, appreciate the help!

*- pike

Saturday 31 March 2007 2:33:58 am

Hi Mauricio

Yes, there's been more post about the memory issue. There are 2 big factors in ImportXMLData: parsing the xml file and creating new objects.

I've loaded files of 900k with a 256Mb memory limit. It was slow, but it worked.
But when creating new objects, every object stays in memory (sortof). This can be improved in the script, but not taken away completely; something happens deeper in the kernel that eats memory for every created object.

>I added: $attribute->setAttribute("data_text",utf8_encode($xml_content_string))
>To the importXMLDatafunctioncollection.php file to import the norwegian letters into the >database.

good tip, thanks.

>Any solutions?
>Commandline? or other fixes.

I dont think commandline will help much (but havent tried). See
http://ez.no/community/forum/developer/memory_leak_on_content_publish
for other possible solutions. No final solution though ...

$2c,
*pike

---------------
The class eZContentObjectTreeNode does.

Fred Prigent

Sunday 22 April 2007 3:11:43 am

Hi,

I would like to use the multi-language aspect of an object in adding a language tag in my xml file.
How does eZPublish manage several languages of an object?Shoud I create a new version?Have you and idea?

Thx

Sorry for my english I'm a french guy too :p

luci lucia

Tuesday 24 April 2007 5:57:02 am

----moderated: spam---

*- pike

Thursday 16 August 2007 8:23:03 am

I uploaded my XMLImport contribution to the contributions. It was originally based on this code, however, reworked back and front quite unrecognizable and in no way compatible anymore :-)

This thread is used to talk about ImportXMLData, so let's keep it like that. If you have questions or comments about XMLImport, add comments to the contribution's page.

http://ez.no/community/contribs/import_export/xmlimport

thanks!
*-pike

---------------
The class eZContentObjectTreeNode does.

Mingxing Chen

Tuesday 25 December 2007 8:15:51 pm

Merry Christmas.

Does this extension work on eZ 4.0?
I tried to import some data, but got these errors:

Error: eZModuleFunctionInfo::execute  	Dec 26 2007 01:35:49

No class 'XMLImportFunctionCollection' available for function 'XMLImport' in module 'XMLImport'

Error: eZTemplate  	12�� 26 2007 10:20:02

Cannot retrieve attribute of a NULL

How can it work on PHP 5?

Please...

Shinetech is the leader in next generation global application outsourcing. Shinetech combines world class technical skills, recognized expertise in advanced development methodologies and industry experience to deliver results for global enterprises. Shinetech utilizes distributed Agile methods for rapid and precise systems development.
------------------------------------------------------------------------------------------
Blog: http://www.mingxing.me
Email: [email protected]

Andy Caiger

Monday 06 September 2010 3:59:12 am

This extension can be made to work with eZ Publish 4.0, but not eZ Publish 4.1 and above. This is because the image import functions use the ezimage class which is deprecated and removed from eZ Publish 4.1. Can anyone suggest how to use the ezimagefile class to replace the functionality of the ezimage class? It seems a shame for this extension to become extinct!

EAB - Integrated Internet Success
Offices in England, France & China.
http://www.eab.co.uk http://www.eab-china.com http://www.eab-france.com

Jérôme Vieilledent

Monday 06 September 2010 4:51:02 am

Hi Andy

You may consider SQLIImport : http://projects.ez.no/sqliimport :)