Merge branch 'release/alpha/master'

Author: coderarjob

Manifest.txt

@@ -2,18 +2,17 @@
 
 ![Screenshot](/docs/images/screenshot_alpha.png)
 
-If you use Kindle to read PDF books or documents, you might have seen that the highlights and notes
-made on the Kindle are not saved on the PDF file itself. This means, that if you take the PDF file
-from your Kindle and read on another device, you will not see those highlights and notes there.
+I sometimes read PDF documents on my Kindle and anyone who does the same, knows that highlights and
+notes taken on the Kindle are not saved on the PDF file itself. This presents a huge problem
+archiving my notes and to redo the highlights and copy the notes manually on to the PDF later on
+is not practical for me.
 
-This software tries to provide a solution. The basis is the Clippings.txt file on your Kindle.
-Kindle saves the page numbers and content of the highlights and notes in the text file. So in
-theory, one can read the Clippings file and reapply the highlights and notes on the PDF separately.
-This software automates the process.
+This software tries to provide a solution. **kpdfsync** reads the ‘My Clippings.txt’ text file for
+the highlights and notes and then applies them in the correct place on a PDF file.
 
-Currently it is in development, so not all the features work or even present. There can also be
-serious bugs present, so I request you to keep backup of the files you give to kpdfsync (the
-clippings file and pdf files)
+Currently it is **under development**, so all the features may not work or be present.
+There can also be serious bugs present, so I **request you to keep backup of the files you give to
+kpdfsync (the clippings file and pdf files)**.
 
 Here is the rough roadmap of the development.
 
@@ -30,7 +29,14 @@ Here is the rough roadmap of the development.
   manually by the user.
 - [X] GUI finalizing for the Alpha release.
 - [X] Debug loggings
-- [X] **Alpha Release**
+- [X] **Alpha Release 1**
+
+----
+
+- [X] Bug Fixes
+- [X] Feature: Fixing common PDF errors.
+- [X] Usage guide for the end user.
+- [X] **Alpha Release 2**
 
 ----
 
@@ -41,7 +47,16 @@ Here is the rough roadmap of the development.
 
 ## Requirements
 - JRE 1.8 or higher
-- Linux, Mac, Windows
+- Linux, Windows
+
+Can probably work on macOS, but I do not own one, so not able to test.
+
+#### For fixing feature to work, you require the following:
+* Linux: poppler-utils
+* Windows: poppler-utils is included with the release.
+
+**Note:** This is a new feature, which can resolve few of the PDF errors, but not all. New fixes
+will be added later on.
 
 ## 3rd-party License
 

README.md

@@ -55,7 +55,10 @@ Kpdfsync
     Highlighting  = 12 67        (Expected, Much more accepatble 
                                   highlighting)
 
-[ ] For some PDF files, org.pdfclown.tools.TextExtractor.extract() is returning null.
+[ ] (GitHub Issue #2)
+    Book: Concrete Mathematics original PDF file.
+
+    For some PDF files, org.pdfclown.tools.TextExtractor.extract() is returning null.
     This is seen with the Concrete Mathematics original PDF file. May be a TrueType font issue.
     Here is the stack trace:
     java.lang.NullPointerException
@@ -87,6 +90,10 @@ Kpdfsync
             at org.pdfclown.tools.TextExtractor.extract(TextExtractor.java:296)
             at coderarjob.kpdfsync.lib.annotator.PdfAnnotatorV1.highlight(PdfAnnotatorV1.java:62)
             at coderarjob.kpdfsync.poc.MainFrame$2.run(MainFrame.java:172)
+    
+    Solution:
+    Running pdftocairo tool (from poppler-utils package), solves this error.
+    Command: pdftocairo -pdf <in pdf file> <out pdf file>
 
 [ ] Highlight is not visible on the output PDF file. This was seen on the Concrete Mathematics
     cropped PDF file.
@@ -103,7 +110,8 @@ Kpdfsync
 
     The times, this exception occures, it occures around the 73% mark.
 
-[ ] EOFException at org.pdfclown.tools.TextExtractor.extract() method. This is seen on
+[ ] (GitHub Issue #1)
+    EOFException at org.pdfclown.tools.TextExtractor.extract() method. This is seen on
     'the_evolution_of_operating_system_cropped.pdf' file. Could also be a font issue.
     Here is the stack trace
 java.lang.RuntimeException: java.io.EOFException
@@ -173,3 +181,60 @@ java.io.EOFException
         at org.pdfclown.tools.TextExtractor.extract(TextExtractor.java:296)
         at coderarjob.kpdfsync.lib.annotator.PdfAnnotatorV1.highlight(PdfAnnotatorV1.java:62)
         at coderarjob.kpdfsync.poc.MainFrame$2.run(MainFrame.java:172)
+
+    Solution:
+    Running pdftocairo tool (from poppler-utils package), solves this error.
+    Command: pdftocairo -pdf <in pdf file> <out pdf file>
+
+[ ] (GitHub Issue #4)
+    Book: MMURTL 
+    org.pdfclown.util.NotImplementedException: LZWDecode
+
+        Stack trace:
+        org.pdfclown.util.NotImplementedException: LZWDecode
+        at org.pdfclown.bytes.filters.Filter.get(Filter.java:74)
+        at org.pdfclown.objects.PdfStream.getBody(PdfStream.java:193)
+        at org.pdfclown.objects.PdfStream.getBody(PdfStream.java:155)
+        at org.pdfclown.documents.contents.Contents$ContentStream.moveNextStream(Contents.java:279)
+        at org.pdfclown.documents.contents.Contents$ContentStream.(Contents.java:86)
+        at org.pdfclown.documents.contents.Contents.load(Contents.java:591)
+        at org.pdfclown.documents.contents.Contents.(Contents.java:366)
+        at org.pdfclown.documents.contents.Contents.wrap(Contents.java:345)
+        at org.pdfclown.documents.Page.getContents(Page.java:571)
+        at org.pdfclown.documents.contents.ContentScanner.(ContentScanner.java:1033)
+        at org.pdfclown.tools.TextExtractor.extract(TextExtractor.java:297)
+        at coderarjob.kpdfsync.lib.annotator.PdfAnnotatorV1.highlight(PdfAnnotatorV1.java:62)
+        at coderarjob.kpdfsync.poc.MainFrame$2.run(MainFrame.java:202)
+        at java.base/java.lang.Thread.run(Thread.java:833)
+
+    Solution:
+    Running pdftocairo tool (from poppler-utils package), solves this error.
+    Command: pdftocairo -pdf <in pdf file> <out pdf file>
+
+[ ] (GitHub Issue #3)
+    Book: https://plan9.io/sys/doc/lexnames.pdf
+
+    Cannot invoke "org.pdfclown.documents.contents.IContentContext.getContents()" 
+    because "contentContext" is null
+
+        Stack Trace:
+        Exception :Cannot invoke "org.pdfclown.documents.contents.IContentContext.getContents()" because "contentContext" is null
+        java.lang.NullPointerException: Cannot invoke "org.pdfclown.documents.contents.IContentContext.getContents()" because "contentContext" is null
+        at org.pdfclown.documents.contents.ContentScanner.(ContentScanner.java:1033)
+        at org.pdfclown.tools.TextExtractor.extract(TextExtractor.java:297)
+        at coderarjob.kpdfsync.lib.annotator.PdfAnnotatorV1.highlight(PdfAnnotatorV1.java:62)
+        at coderarjob.kpdfsync.poc.MainFrame$2.run(MainFrame.java:202)
+        at java.base/java.lang.Thread.run(Thread.java:833)
+
+[ ] Index Out Of Bounds in PdfAnnotatorV1
+    PDF souce :/home/coder/kpdfsync/test-files/Books/Classic Operating Systems_ From Batch Processing To Distributed Systems_cropped.pdf
+
+      Exception :index -1, length 0
+        java.lang.StringIndexOutOfBoundsException: index -1, length 0
+        at java.base/java.lang.String.checkIndex(String.java:4560)
+        at java.base/java.lang.AbstractStringBuilder.deleteCharAt(AbstractStringBuilder.java:970)
+        at java.base/java.lang.StringBuilder.deleteCharAt(StringBuilder.java:298)
+        at coderarjob.kpdfsync.lib.annotator.PdfAnnotatorV1.doHighlight(PdfAnnotatorV1.java:96)
+        at coderarjob.kpdfsync.lib.annotator.PdfAnnotatorV1.highlight(PdfAnnotatorV1.java:65)
+        at coderarjob.kpdfsync.poc.MainFrame$2.run(MainFrame.java:201)
+      at java.base/java.lang.Thread.run(Thread.java:833)

TODO

@@ -1,20 +1,35 @@
 #!/bin/sh
+# -----------------------------------------------------------------------------
+# Compiles .java files and version string creator script               kpdfsync
+# -----------------------------------------------------------------------------
+# This script is responsible for taking the source code and creating a complete
+# runnable program.
+#
+# Creates file system and then builds kpdfsync. It creates the version string
+# for each of the packages and puts them to their respective app.settings
+# files. It packs the resource files for each of the packages in the end.
+# -----------------------------------------------------------------------------
 
 BIN_DIR=build/classes
 
 mkdir -p $BIN_DIR
 rm -rf $BIN_DIR/*
 
+# -----------------------------------------------------------------------------
 # Remove trailing spaces from java source files
+# -----------------------------------------------------------------------------
 find src -name "*.java" -exec sed -i s/\ \*$//g {} \; || exit
 
+# -----------------------------------------------------------------------------
+# Building
+# -----------------------------------------------------------------------------
 export CLASSPATH="lib/pdfclown.jar:$BIN_DIR"
-
 JDK_VER_TARGET=8
 
 # Build AJL
 javac --release $JDK_VER_TARGET -Xlint -d "$BIN_DIR/" \
-      src/coderarjob/ajl/file/*.java || exit
+      src/coderarjob/ajl/file/*.java \
+      src/coderarjob/ajl/*.java || exit
 
 # Build Pattern Matcher
 javac --release $JDK_VER_TARGET -Xlint -d "$BIN_DIR/" \
@@ -34,20 +49,51 @@ javac --release $JDK_VER_TARGET -Xlint -d "$BIN_DIR/" \
 
 # Build POC
 javac --release $JDK_VER_TARGET -Xlint -d "$BIN_DIR/" \
-      src/coderarjob/kpdfsync/poc/*.java || exit
+      src/coderarjob/kpdfsync/poc/*.java \
+      src/coderarjob/kpdfsync/poc/pdffixes/*.java \
+      || exit
 
+# -----------------------------------------------------------------------------
 # Copy resources
+# -----------------------------------------------------------------------------
+cp -r src/coderarjob/ajl/res $BIN_DIR/coderarjob/ajl || exit
+cp -r src/coderarjob/kpdfsync/lib/res $BIN_DIR/coderarjob/kpdfsync/lib || exit
 cp -r src/coderarjob/kpdfsync/poc/res $BIN_DIR/coderarjob/kpdfsync/poc || exit
 
+# -----------------------------------------------------------------------------
+# Replace placeholder information in resource files.
+# -----------------------------------------------------------------------------
+buildid=$(date +%y%m%d)
+find $BIN_DIR -type f -name app.settings \
+    -exec sed -i "s/<build>/$buildid/g" {} \;
+
+command -v git > /dev/null && (
+    path=coderarjob/ajl
+    commitid=$(git log --format="%h" -n 1 src/$path)
+    find $BIN_DIR/$path -type f -name app.settings \
+        -exec sed -i "s/<commitid>/$commitid/g" {} \;
+
+    path=coderarjob/kpdfsync/lib
+    commitid=$(git log --format="%h" -n 1 src/$path)
+    find $BIN_DIR/$path -type f -name app.settings \
+        -exec sed -i "s/<commitid>/$commitid/g" {} \;
+
+    path=coderarjob/kpdfsync/poc
+    commitid=$(git log --format="%h" -n 1 src/$path)
+    find $BIN_DIR/$path -type f -name app.settings \
+        -exec sed -i "s/<commitid>/$commitid/g" {} \;
+) || exit
+
+# -----------------------------------------------------------------------------
 # Generate tags file
-ctags --recurse ./src || exit
-
-# Run
-GTKLOOK="com.sun.java.swing.plaf.gtk.GTKLookAndFeel"
-MOTIFLOOK="com.sun.java.swing.plaf.motif.MotifLookAndFeel"
-METALLOOK="javax.swing.plaf.metal.MetalLookAndFeel"
-WINLOOK="com.sun.java.swing.plaf.windows.WindowsLookAndFeel"
-NIMBUSLLOOK="com.sun.java.swing.plaf.nimbus.NimbusLookAndFeel"
-LOOK=$METALLOOK
-#java -Dswing.defaultlaf=$LOOK \
-java coderarjob.kpdfsync.poc.Main
+# -----------------------------------------------------------------------------
+command -v ctags > /dev/null && (
+  ctags --recurse ./src || exit
+)
+
+# -----------------------------------------------------------------------------
+# Call pack.sh, to generate the jar files.
+# -----------------------------------------------------------------------------
+cd build
+./pack.sh
+cd -

build.sh

@@ -0,0 +1,2 @@
+Main-Class: coderarjob.kpdfsync.poc.Main
+Class-Path: pdfclown.jar ajl.jar libkpdfsync.jar

build/Manifest.txt

@@ -0,0 +1,8 @@
+:: -----------------------------------------------------------------------------
+:: Windows Start script                                                 kpdfsync
+:: -----------------------------------------------------------------------------
+:: Changes directory to the application path and runs kpdfsync
+:: -----------------------------------------------------------------------------
+@echo off
+set BASEDIR=%~dp0
+start /D "%BASEDIR%\bin" /B java -jar kpdfsync.jar

build/kpdfsync.bat

@@ -0,0 +1,9 @@
+#!/bin/sh
+# -----------------------------------------------------------------------------
+# Unix Start script                                                    kpdfsync
+# -----------------------------------------------------------------------------
+# Changes directory to the application path and runs kpdfsync
+# -----------------------------------------------------------------------------
+BASEDIR=$(dirname $0)
+cd "$BASEDIR/bin"
+java -jar kpdfsync.jar&

build/kpdfsync.sh

@@ -0,0 +1,59 @@
+#!/bin/bash
+# -----------------------------------------------------------------------------
+# Package creation script                                              kpdfsync
+# -----------------------------------------------------------------------------
+# This script is responsible for packing all the java .class files,
+# dependencies and scripts and place them in correct places.
+#
+# This placement also ensures, that when distributed on different systems,
+# kpdfsync can be run using the two helper scripts, kpdfsync.sh and
+# kpdfsync.bat.
+# -----------------------------------------------------------------------------
+
+echo :: Building packages
+
+# -----------------------------------------------------------------------------
+# Create destination folder
+# -----------------------------------------------------------------------------
+DIST_DIR=kpdfsync
+rm -rf $DIST_DIR       || exit
+mkdir -p $DIST_DIR/bin || exit
+
+pushd classes
+
+# -----------------------------------------------------------------------------
+# Create .jar files for individual packages.
+# -----------------------------------------------------------------------------
+
+# Package Arjob's Java Library
+echo :: Building ajl.jar
+jar cf ajl.jar coderarjob/ajl || exit
+
+# Package kpdfsync library
+echo :: Building libkpdfsync.jar
+jar cf libkpdfsync.jar coderarjob/kpdfsync/lib || exit
+
+# Package kpdfsync gui
+echo :: Building kpdfsync.jar
+jar cfm kpdfsync.jar ../Manifest.txt coderarjob/kpdfsync/poc || exit
+popd
+
+# -----------------------------------------------------------------------------
+# Move .jar files and scripts to proper places under the kpdfsync dist folder.
+# -----------------------------------------------------------------------------
+
+# Move the jar files to kpdfsync/bin folder.
+mv classes/*.jar ./$DIST_DIR/bin        || exit
+cp ../lib/pdfclown.jar ./$DIST_DIR/bin  || exit
+cp -r ../tools ./$DIST_DIR/bin          || exit
+
+# Copy the kpdfsync.sh and kpdfsync.bat to kpdfsync dist folder.
+cp kpdfsync.sh ./$DIST_DIR              || exit
+cp kpdfsync.bat ./$DIST_DIR             || exit
+
+# Copy LICENSE Readme.md and HowTo.pdf to kpdfsync dist folder.
+cp ../docs/QuickReference.pdf ./$DIST_DIR || exit
+cp ../README.md ./$DIST_DIR               || exit
+cp ../LICENSE ./$DIST_DIR                 || exit
+
+echo :: Building packages completed