Go Back   MarcomCentral (PTI) and FusionPro User Communities > Software-Related Talk > FusionPro® VDP Creator

Notices

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old January 29th, 2009, 09:30 AM
ppachen ppachen is offline
Forum n00b
 
Join Date: Jan 2009
Posts: 2
Exclamation Data Character Issues

I'm running into issues with the carriage return (Enter) and ampersand (&). I typically see the carriage return used in an Address 1 field, separating two lines that should have been placed into Address 1 and Address 2. Once I compose, the rest of the address will be missing after the carriage return.

The ampersand only causes an issue when it is placed next to two letters. "D & B" will not cause a problem but "D&B" will make the "&B" disappear in the final PDF output.

I would like to have a fix in FusionPro for these issues, although I'm open to ideas for correcting the problem within Excel.

Oh, and I'm using FusionPro v4.2P1d.

Thanks!
Reply With Quote
  #2  
Old January 29th, 2009, 10:01 AM
LeberMac's Avatar
LeberMac LeberMac is offline
Junior Community Member
 
Join Date: Jan 2009
Location: Milwaukee, WI
Posts: 25
Default Re: Data Character Issues

I've seen this happen as well, if you replace:
&
with...
&

It seems to work - It looks wrong in the preview, but, upon composing, it works correctly.
Did this using FusionPro 5.8, though. Dunno if it will work in 4.2...
__________________
"I say we take off and nuke the entire site from orbit. It's the only way to be sure."
Reply With Quote
  #3  
Old January 29th, 2009, 05:43 PM
Dan Korn's Avatar
Dan Korn Dan Korn is offline
FusionPro Senior Engineer / Forum Moderator
 
Join Date: Aug 2008
Location: Chicago, IL
Posts: 4,540
Default Re: Data Character Issues

This problem is exactly what the NormalizeEntities function was designed to resolve. There are several things you can do:

Use the NormalizeEntities function in a rule with the "Treat returned strings as tagged text" box checked, like so:
Code:
return NormalizeEntities(Field("MyFieldName"));
Or, you can simply return the field value in a rule with the "Treat returned strings as tagged text" box unchecked:
Code:
return Field("MyFieldName");
Or, you can uncheck the "Treat field values as tagged text" box on the "Flat File" tab of the Data Source Wizard.

In FusionPro 6.0, the new TaggedTextFromRaw function supercedes the NormalizeEntities function and has more robust support for various characters which can cause problems for the tagged markup parser.
__________________
Dan Korn
FusionPro Developer / JavaScript Guru / Forum Moderator
PTI Marketing Technologies | Printable | MarcomCentral
LinkedIn

I am a not a Support engineer, and this forum is not a substitute for Support. My participation on this forum is primarily as a fellow user (and a forum moderator). I am happy to provide help and answers to questions when I can; however, there is no guarantee that I, or anyone else on this forum, will be able to answer all questions or fix any problems. If I ask for files to clarify an issue, I might not be able to look at them personally. I am not able to answer private messages, emails, or phone calls unless they go through proper Support channels. Please direct any sales or pricing questions to your salesperson or inquiries@marcom.com.

Complex template-building questions, as well as all installation and font questions or problems, should be directed to FusionProSupport@marcom.com. Paid consulting work may be required to fulfill your template-building needs.

This is a publicly viewable forum. Please DO NOT post fonts, or other proprietary content, to this forum. Also, please DO NOT post any "live" data with real names, addresses, or any other personal, private, or confidential data.

Please include the specific versions of FusionPro, Acrobat, and your operating system in any problem reports or help requests. I recommend putting this information in your forum signature. Please also check your composition log (.msg) file for relevant error or warning messages.

Please post questions specific to the MarcomCentral Enterprise and Web-to-Print applications in the MarcomCentral forum. Click here to request access. Or contact your Business Relationship Manager (BRM/CPM) for assistance.

Please direct any questions specific to EFI's Digital StoreFront (DSF) to EFI support.

How To Ask Questions The Smart Way

The correct spellings are JavaScript, FusionPro, and MarcomCentral (each with two capital letters and no spaces). Acceptable abbreviations are JS, FP, and MC (or MCC). There is no "S" at the end of "Expression" or "Printable"! The name of the product is FusionPro, not "Fusion". "Java" is not is not the same as JavaScript.

Check out the JavaScript Guide and JavaScript Reference! FusionPro 8.0 and newer use JavaScript 1.7. Older versions use JavaScript 1.5.

return "KbwbTdsjqu!spdlt\"".replace(/./g,function(w){return String.fromCharCode(w.charCodeAt()-1)});
Reply With Quote
  #4  
Old January 30th, 2009, 07:14 AM
MartyrSyx's Avatar
MartyrSyx MartyrSyx is offline
Regular Contributor
 
Join Date: Sep 2008
Location: Westerville, Ohio
Posts: 70
Default Re: Data Character Issues

Dan,

For some reason I can't find documentation on the new features in 6.0 on how to use them. I don't know if I'm missing it or looking in the wrong place. Do you know where I can look?
__________________
Brad Mather
WestCamp
Reply With Quote
  #5  
Old January 30th, 2009, 10:47 AM
Dan Korn's Avatar
Dan Korn Dan Korn is offline
FusionPro Senior Engineer / Forum Moderator
 
Join Date: Aug 2008
Location: Chicago, IL
Posts: 4,540
Default Re: Data Character Issues

Quote:
Originally Posted by MartyrSyx View Post
For some reason I can't find documentation on the new features in 6.0 on how to use them. I don't know if I'm missing it or looking in the wrong place. Do you know where I can look?
http://forums.printable.com/showthread.php?t=329
http://forums.printable.com/showthread.php?t=324
__________________
Dan Korn
FusionPro Developer / JavaScript Guru / Forum Moderator
PTI Marketing Technologies | Printable | MarcomCentral
LinkedIn

I am a not a Support engineer, and this forum is not a substitute for Support. My participation on this forum is primarily as a fellow user (and a forum moderator). I am happy to provide help and answers to questions when I can; however, there is no guarantee that I, or anyone else on this forum, will be able to answer all questions or fix any problems. If I ask for files to clarify an issue, I might not be able to look at them personally. I am not able to answer private messages, emails, or phone calls unless they go through proper Support channels. Please direct any sales or pricing questions to your salesperson or inquiries@marcom.com.

Complex template-building questions, as well as all installation and font questions or problems, should be directed to FusionProSupport@marcom.com. Paid consulting work may be required to fulfill your template-building needs.

This is a publicly viewable forum. Please DO NOT post fonts, or other proprietary content, to this forum. Also, please DO NOT post any "live" data with real names, addresses, or any other personal, private, or confidential data.

Please include the specific versions of FusionPro, Acrobat, and your operating system in any problem reports or help requests. I recommend putting this information in your forum signature. Please also check your composition log (.msg) file for relevant error or warning messages.

Please post questions specific to the MarcomCentral Enterprise and Web-to-Print applications in the MarcomCentral forum. Click here to request access. Or contact your Business Relationship Manager (BRM/CPM) for assistance.

Please direct any questions specific to EFI's Digital StoreFront (DSF) to EFI support.

How To Ask Questions The Smart Way

The correct spellings are JavaScript, FusionPro, and MarcomCentral (each with two capital letters and no spaces). Acceptable abbreviations are JS, FP, and MC (or MCC). There is no "S" at the end of "Expression" or "Printable"! The name of the product is FusionPro, not "Fusion". "Java" is not is not the same as JavaScript.

Check out the JavaScript Guide and JavaScript Reference! FusionPro 8.0 and newer use JavaScript 1.7. Older versions use JavaScript 1.5.

return "KbwbTdsjqu!spdlt\"".replace(/./g,function(w){return String.fromCharCode(w.charCodeAt()-1)});
Reply With Quote
  #6  
Old January 30th, 2009, 01:16 PM
ppachen ppachen is offline
Forum n00b
 
Join Date: Jan 2009
Posts: 2
Default Re: Data Character Issues

Thanks Dan! I was able to fix the issue with the ampersand. I'm unfortunately still running into issues with the carriage return. I tried both methods in v4.2P1d but neither seemed to work. The Preview Record Selector shows "<Missing>" for all fields after the offending carriage return. Ideally I would like the carriage return to be read so that each line in the single field is placed as a separate line in the output PDF.

Will an upgrade to v6 correct this problem (assuming I'm also using your suggested fixes)? Or does the carriage return inherently cause issues?
Reply With Quote
  #7  
Old January 30th, 2009, 02:33 PM
Dan Korn's Avatar
Dan Korn Dan Korn is offline
FusionPro Senior Engineer / Forum Moderator
 
Join Date: Aug 2008
Location: Chicago, IL
Posts: 4,540
Default Re: Data Character Issues

Quote:
Originally Posted by ppachen View Post
Will an upgrade to v6 correct this problem (assuming I'm also using your suggested fixes)? Or does the carriage return inherently cause issues?
Yes, the carriage return inherently causes issues, and No, simply upgrading to 6.0 won't resolve them.

Since you mention Excel, I assume that you're dumping to a flat input data file, probably either comma-delimited (CSV) or tab-delimited. The problem is that in flat file data, a newline character denotes a new record, i.e. a new "line" of data, by definition. So newlines are not valid within field values in delimited data. (If you're using tagged markup input, then newlines are silently stripped out as whitespace and ignored.) Nothing about this has changed in FusionPro 6.0, and I can't foresee it changing in any subsequent version. Newlines are row delimiters, and there's really no getting around that.

This situation is further complicated on Windows by the fact that an embedded newline is actually composed of both a newline (ASCII 10, or "\n" in JavaScript) character and a carriage-return (ASCII 13, or "\r" in JavaScript) character, either of which will be treated as an end-of-line character (record delimiter) by FusionPro. Line-ending variations on different computing platforms have been a source of headaches for decades:
http://en.wikipedia.org/wiki/Newline#Common_problems

So, you'll need to somehow tell Excel to not output extra newlines or carriage returns to the flat data file. If you want to preserve them so that they actually cause line breaks in the FusionPro output, your best bet is to convert them to <br> tags. You'll need to use a VBA macro to do the conversion and make sure you get all of the newline and carriage-return characters. Here's a macro I found on Google and modified which should do what you need:
Code:
 Sub ReplaceCR()
    Dim ws As Worksheet
    For Each ws In Worksheets
        With ws.Cells
            .Replace vbCr, "<br>"
            .Replace vbLf, ""
            .Replace vbCrLf, ""
        End With
    Next
    MsgBox "Done"
End Sub
Again, this is an Excel VBA macro, not anything you can use in FusionPro directly. You can find more examples with a search like this:
http://google.com/search?q=excel+rep...urn%22+newline

Once you've converted these characters in Excel, you can export to CSV and then tell FusionPro to treat the field values as tagged text so that the <br> tags get processed properly.

But unfortunately, we're not done yet, because now we're back to the problems with the ampersands and such, since now those are treated as markup again. The way out of this dilemma is to modify the VBA macro above to output something else that you can replace with a <br> tag at composition time in FusionPro. So the Excel macro could look like this:
Code:
 Sub ReplaceCR()
    Dim ws As Worksheet
    For Each ws In Worksheets
        With ws.Cells
            .Replace vbCr, "***NEWLINE***"
            .Replace vbLf, ""
            .Replace vbCrLf, ""
        End With
    Next
    MsgBox "Done"
End Sub
Then you can do something like this in a rule:
Code:
return ReplaceSubstring(NormalizeEntities(Field("MyFieldName")), "***NEWLINE***", "<br>");
You'll need to do the replacement above for each field in the job. If you're using a version of FusionPro previous to 6.0, you'll have to create a set of rules like this, one rule for each data field, with the same name as the field.

If you're using FusionPro 6.0, you can do it all in one shot in OnRecordStart like so:
Code:
for (var i in FusionPro.Fields)
{
  FusionPro.Composition.AddVariable(i, ReplaceSubstring(TaggedTextFromRaw(Field(i)), "***NEWLINE***", "<br>"));
}
The underlying problem here is that, even though Excel is The World's Most Popular Database (TM), it's not really a database at all, and the step of exporting to a flat file doesn't handle embedded newlines well. That said, your other option, if you're on Windows, is to extract the data directly from Excel to FusionPro with ODBC instead of going through a CSV or a tab-delimited file.
__________________
Dan Korn
FusionPro Developer / JavaScript Guru / Forum Moderator
PTI Marketing Technologies | Printable | MarcomCentral
LinkedIn

I am a not a Support engineer, and this forum is not a substitute for Support. My participation on this forum is primarily as a fellow user (and a forum moderator). I am happy to provide help and answers to questions when I can; however, there is no guarantee that I, or anyone else on this forum, will be able to answer all questions or fix any problems. If I ask for files to clarify an issue, I might not be able to look at them personally. I am not able to answer private messages, emails, or phone calls unless they go through proper Support channels. Please direct any sales or pricing questions to your salesperson or inquiries@marcom.com.

Complex template-building questions, as well as all installation and font questions or problems, should be directed to FusionProSupport@marcom.com. Paid consulting work may be required to fulfill your template-building needs.

This is a publicly viewable forum. Please DO NOT post fonts, or other proprietary content, to this forum. Also, please DO NOT post any "live" data with real names, addresses, or any other personal, private, or confidential data.

Please include the specific versions of FusionPro, Acrobat, and your operating system in any problem reports or help requests. I recommend putting this information in your forum signature. Please also check your composition log (.msg) file for relevant error or warning messages.

Please post questions specific to the MarcomCentral Enterprise and Web-to-Print applications in the MarcomCentral forum. Click here to request access. Or contact your Business Relationship Manager (BRM/CPM) for assistance.

Please direct any questions specific to EFI's Digital StoreFront (DSF) to EFI support.

How To Ask Questions The Smart Way

The correct spellings are JavaScript, FusionPro, and MarcomCentral (each with two capital letters and no spaces). Acceptable abbreviations are JS, FP, and MC (or MCC). There is no "S" at the end of "Expression" or "Printable"! The name of the product is FusionPro, not "Fusion". "Java" is not is not the same as JavaScript.

Check out the JavaScript Guide and JavaScript Reference! FusionPro 8.0 and newer use JavaScript 1.7. Older versions use JavaScript 1.5.

return "KbwbTdsjqu!spdlt\"".replace(/./g,function(w){return String.fromCharCode(w.charCodeAt()-1)});
Reply With Quote
Reply

Tags
ampersand, carriage, enter, return, sign

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:24 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
(c) 2011, PTI Marketing Technologies™, Inc.