Categories
DataBase Development Diary Development

MySQL InnoDB Data Recovery

A few days ago I had something like 80 tabs open in Chrome and Safari, Atom was running (with several tabs open too), numerous other applications and (listen, listen!) at a certain point I decided that I wanted to “play” with Garage Band and his “cheerful” synthesizers. Of course, I did it without thinking about closing some of the open browser tabs. I couldn’t live without them!
As a result the Macbook crashed. A crash like this has not happened for at least one year.

So far so good! Except that after this event MySQL could no longer start: shutdown itself shortly after launch. Looking at the logs I came to the conclusion that some table probably got corrupted. I tried to upgrade MySQL to a newer version but without success.
I have about 50 databases, installed locally, for my Web development projects and I do not do daily backups (this has been a good lesson to decide to plan a daily, automatic solution) and I opt for them when I think it’s the case. I had a backup copy of the local version of this site dating back about 4 days ago. It would still have been enough but I wanted to try and restore the state it was in just before the accident. That’s why I tried to find a solution on the internet to recover data.

First of all I have to premise that the database tables (mainly local installations of WordPress) were created using InnoDB as an engine. I do not know much about it, but Oracle published some useful utilities among which there is a very useful application called mysqlfrm and it’s part of a package, MySQL Utilities, which can be downloaded from this address: https://downloads.mysql.com/archives/utilities/. The installation is very simple but before you can use them you must also remember to install another package on which they depend: MySQL Connector/Python. Once you have installed this package you will have everything you need to proceed with data recovery.

Note

According to the official website, MySQL Utilities should be installed with MySQL Workbench but I had already installed it on the Macbook and I could not find mysqlfrm. Now I’m trying to install the latest release of MySQL Workbench and we’ll see…
MySQL WorkBench - MySQL Utilities
MySQL WorkBench – MySQL Utilities

Once you have installed these two programs you could also perform the whole procedure manually as explained in countless articles on the Web but I decided to further simplify your life. I have just finished writing a Shell Script between yesterday and today (compatible with all Unix and Unix Like environments) that does everything by itself. You can download it from here: recover_innodb

To run it is easy, open the Terminal application and run this command:

sudo ./recover_innodb.sh -d dest_db_name -b dest_data_dir -s src_db_name -c src_data_dir

The options for now are:

  • -d, to indicate the name of the resulting database,
  • -b, to indicate the path of the destination “data” directory,
  • -s, to indicate the name of the original database,
  • -c, to indicate the path of the original “data” directory.

If you are not wrong writing the paths the script should work correctly.

Good data recovery!

Categories
Development Diary Development

Seven days

As in the famous song by Sting, Seven days, one of my favorites, it took seven days (nearly 8, actually) to correct and refine the source code of the Full Pipe Umbrella theme (and it is not over yet). The WordPress theme I’m using for this Web site is actually a Child Theme of Flat-Theme written by Shapebootstrap.net.

The work done in these 7 days, culminated in the past 3 days, led me to perform 14 commits and numerous small fixes to the source code.

Categories
Development Diary Development Web Development

Bootstrap Installation 4.1.0: in case of error!

In the world of Open Source things never go as they should go! And even the seemingly innocuous task of installing the sources and the now consolidated Bootstrap dependencies may hold some surprises. I downloaded the ZIP archive of Bootstrap sources from their official download page.

After extracting the archive into a folder of my choice I started the installation of dependencies as instructed. And, as you can see for yourself, I have had a few complaints.

$ npm install
events.js:183░░░░░░⸩ ⠹ extract:postcss-cli: sill extract bundlesize@0.16.0
      throw er; // Unhandled 'error' event
      ^

Error: write after end
    at writeAfterEnd (_stream_writable.js:236:12)
    at PassThrough.Writable.write (_stream_writable.js:287:5)
    at PassThrough.Writable.end (_stream_writable.js:553:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/pacote/lib/extract-stream.js:19:41)
    at emitOne (events.js:121:20)
    at ReadEntry.emit (events.js:211:7)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:287:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:240:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:153:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at emitOne (events.js:116:13)
    at Unzip.emit (events.js:211:7)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:287:25)
events.js:183░░░░░░⸩ ⠼ extract:yargs-parser: sill extract yargs-parser@4.2.1
      throw er; // Unhandled 'error' event
      ^

Error: write after end
    at writeAfterEnd (_stream_writable.js:236:12)
    at PassThrough.Writable.write (_stream_writable.js:287:5)
    at PassThrough.Writable.end (_stream_writable.js:553:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/pacote/lib/extract-stream.js:19:41)
    at emitOne (events.js:121:20)
    at ReadEntry.emit (events.js:211:7)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:287:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:240:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:153:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at emitOne (events.js:116:13)
    at Unzip.emit (events.js:211:7)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:287:25)
WARN tar ENOENT: no such file or directory, open '/Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/.staging/socket.io-client-c6efd21f/dist/socket.io.slim.js'
WARN tar ENOENT: no such file or directory, open '/Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/.staging/rollup-f2b9ab81/dist/rollup.es.js'
npm WARN bootstrap@4.1.0 requires a peer of jquery@1.9.1 - 3 but none is installed. You must install peer dependencies yourself.

npm ERR! cancel after 1 retries!

As I said, with open source software such things happen frequently. The programmer’s job is made, in fact, at least 50% of his time, the resolution of this kind of problems (that have nothing to do with the actual programming). So, I was armed with patience (lowering) and I tried to figure it out.

As you know (if already you delight of these things) Node is a JavaScript interpreter, equipped with APIs that extend its capabilities, which allows the execution of JavaScript scripts in the system environment. NPM is the Node Package Manager and has become an ubiquitous tool, a de facto standard for the development of Web software and the management of packages and dependencies.

However! Following divine insight I thought that the problem of my installation was due to some folder permissions. So I went to look around the place where they are installed all of the packages node, or in the directory /usr/local/lib/node_modules. Once I checked them in the configuration of the content permissions.

 

$ ls -lG node_modules/
total 0
drwxr-xr-x   8 your_user_name  staff  272 12 Ott  2017 bower
drwxr-xr-x  14 your_user_name  staff  476  7 Nov 11:33 browserify
drwxr-xr-x  22 your_user_name  staff  748 26 Mar  2017 cordova
drwxr-xr-x   8 your_user_name  staff  272 26 Mar  2017 css-select
drwxr-xr-x   6 your_user_name  staff  204 26 Mar  2017 css-what
drwxr-xr-x  16 your_user_name  staff  544 26 Mar  2017 generator-wp-bones
drwxr-xr-x   9 your_user_name  staff  306 26 Mar  2017 graceful-fs
drwxr-xr-x   9 your_user_name  staff  306  7 Nov 13:46 grunt-cli
drwxr-xr-x   9 root            staff  306 12 Gen 18:54 js-beautify
drwxr-xr-x  11 your_user_name  staff  374 12 Ott  2017 jshint
drwxrwxr-x   7 your_user_name  staff  238 16 Feb  2017 json-beautifier
drwxr-xr-x  24 your_user_name  staff  816  7 Nov 15:28 less
drwxr-xr-x   6 root            staff  204 12 Gen 18:54 less-watch-compiler
drwxr-xr-x  14 your_user_name  staff  476 13 Nov 17:23 live-server
drwxr-xr-x   7 your_user_name  staff  238 12 Ott  2017 minimatch
drwxr-xr-x  27 root            staff  918  4 Apr 11:15 npm
drwxr-xr-x  11 your_user_name  staff  374 26 Mar  2017 npmconf
drwxr-xr-x  19 your_user_name  staff  646 12 Ott  2017 phonegap
drwxr-xr-x  12 your_user_name  staff  408 12 Ott  2017 pug
drwxr-xr-x  16 your_user_name  staff  544 16 Gen 19:14 uuid
drwxr-xr-x   6 your_user_name  staff  204 26 Mar  2017 yo

 

And in fact, some directories are owned by root. That could create some inconvenience. So I changed the owner of all the folders in this way.

$ sudo chown -R your_user_name node_modules

After making this change, I tried again to run npm install and everything went well, except for some error message that you can ignore (I think).

$ npm install
events.js:183░░░░░░⸩ ⠸ extract:minimist: sill extract minimist@1.2.0
      throw er; // Unhandled 'error' event
      ^

Error: write after end
    at writeAfterEnd (_stream_writable.js:236:12)
    at PassThrough.Writable.write (_stream_writable.js:287:5)
    at PassThrough.Writable.end (_stream_writable.js:553:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/pacote/lib/extract-stream.js:19:41)
    at emitOne (events.js:121:20)
    at ReadEntry.emit (events.js:211:7)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:287:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:240:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:153:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at emitOne (events.js:116:13)
    at Unzip.emit (events.js:211:7)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:287:25)

> fsevents@1.1.3 install /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/fsevents
> node install

[fsevents] Success: "/Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/fsevents/lib/binding/Release/node-v57-darwin-x64/fse.node" already installed
Pass --update-binary to reinstall or --build-from-source to recompile

> uws@9.14.0 install /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/uws
> node-gyp rebuild > build_log.txt 2>&1 || exit 0


> iltorb@1.3.10 install /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/iltorb
> detect-libc prebuild-install || node-gyp rebuild

prebuild-install info begin Prebuild-install version 2.5.1
prebuild-install info looking for local prebuild @ prebuilds/iltorb-v1.3.10-node-v57-darwin-x64.tar.gz
prebuild-install info looking for cached prebuild @ /Users/your_user_name/.npm/_prebuilds/https-github.com-MayhemYDG-iltorb-releases-download-v1.3.10-iltorb-v1.3.10-node-v57-darwin-x64.tar.gz
prebuild-install http request GET https://github.com/MayhemYDG/iltorb/releases/download/v1.3.10/iltorb-v1.3.10-node-v57-darwin-x64.tar.gz
prebuild-install http 200 https://github.com/MayhemYDG/iltorb/releases/download/v1.3.10/iltorb-v1.3.10-node-v57-darwin-x64.tar.gz
prebuild-install info downloading to @ /Users/your_user_name/.npm/_prebuilds/https-github.com-MayhemYDG-iltorb-releases-download-v1.3.10-iltorb-v1.3.10-node-v57-darwin-x64.tar.gz.29945-29ba072d5c85f.tmp
prebuild-install info renaming to @ /Users/your_user_name/.npm/_prebuilds/https-github.com-MayhemYDG-iltorb-releases-download-v1.3.10-iltorb-v1.3.10-node-v57-darwin-x64.tar.gz
prebuild-install info unpacking @ /Users/your_user_name/.npm/_prebuilds/https-github.com-MayhemYDG-iltorb-releases-download-v1.3.10-iltorb-v1.3.10-node-v57-darwin-x64.tar.gz
prebuild-install info unpack resolved to /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/iltorb/build/bindings/iltorb.node
prebuild-install info unpack required /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/iltorb/build/bindings/iltorb.node successfully
prebuild-install info install Successfully installed prebuilt binary!

> node-sass@4.7.2 install /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/node-sass
> node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.7.2/darwin-x64-57_binding.node
Download complete  ⸩ ⠋ :
Binary saved to /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/node-sass/vendor/darwin-x64-57/binding.node
Caching binary to /Users/your_user_name/.npm/node-sass/4.7.2/darwin-x64-57_binding.node

> node-sass@4.7.2 postinstall /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/node-sass
> node scripts/build.js

Binary found at /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/node-sass/vendor/darwin-x64-57/binding.node
Testing binary
Binary is fine

> nodemon@1.17.2 postinstall /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/nodemon
> node -e "console.log('\u001b[32mLove nodemon? You can now support the project via the open collective:\u001b[22m\u001b[39m\n > \u001b[96m\u001b[1mhttps://opencollective.com/nodemon/donate\u001b[0m\n')" || exit 0

Love nodemon? You can now support the project via the open collective:
 > https://opencollective.com/nodemon/donate


> sinon@4.4.6 postinstall /Users/your_user_name/Sites/bootstrap_cv_15-04-2018/node_modules/sinon
> node -e "console.log('\u001b[32mLove sinon? You can now support the project via the open collective:\u001b[22m\u001b[39m\n > \u001b[96m\u001b[1mhttps://opencollective.com/sinon/donate\u001b[0m\n')" || exit 0

Love sinon? You can now support the project via the open collective:
 > https://opencollective.com/sinon/donate

npm WARN bootstrap@4.1.0 requires a peer of jquery@1.9.1 - 3 but none is installed. You must install peer dependencies yourself.

added 1600 packages from 1445 contributors in 62.771s
Categories
Development Diary Development Web Development

Atom

Sorry, this entry is only available in Italian. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

Sembra che Atom e Git, su Windows 10 si contendano i permessi delle cartelle. Probabilmente quelle correntemente aperte da Atom.

Mentre stavo lavorando su un’installazione di WordPress in locale (sul computer su cui lavoro), ottenuta installando XAMPP e Bitnami WordPress, ho tentato di eseguire l’aggiornamento di un Plugin che, naturalmente, è fallita. Tutto questo mentre avevo aperto su Atom uno dei file sorgente dello stesso Plugin. Dopo questo fallimento ho anche tentato di ripristinare l’HEAD di Git, in modo da evitare pasticci, ma senza successo.

git checkout -- '*'

Il comando precedente falliva miseramente.

Dopo vari tentativi (fra i quali l’esecuzione dei comandi suggeriti in questa risposta https://superuser.com/a/629362, che sono naturalmente miseramente falliti) ho chiuso per caso Atom. E la cartella che, prima, non riuscivo neppure a cancellare è scomparsa di colpo.

A quel punto mi è venuto un sospetto e ho riprovato a eseguire il comando Git che, sta volta, ha avuto successo. E, subito dopo, sono riuscito anche ad aggiornare il Plugin dal pannello di amministrazione di WordPress senza problemi.

Boh! Magie del mondo Microsoft Windows! Che su piattaforme UNIX non mi erano mai accadute.

Categories
Blender Scripting (BPY) Computer Graphics Development Diary Development

Blender Python script: how to position a plane to apparently fit camera frame

In the past days I faced another challenge while working to my Blender 3D experiments.
An idea came to me when I asked myself: « Which is the optimal resolution of a texture, depending on the distance of an object from the camera? » and I started to split the problem in smaller tasks.

The first answer was: « Assume that we are watching a square plane, which dimensions apparently fit the frame of the camera. And assume that the texture perfectly fits the borders of the plane (a common situation, obtained by executing “Unwrap” command, in “Editing” mode). The condition, required to obtain such result, is to place the plane at a “special” distance from the camera (distance that I will call Critical Distance). At that distance the plane texture can have exactly the same resolution of the final render frame. Beyond the Critical Distance, the texture resolution can be lower. At distances lower then the CD, texture must have higher resolution then the render frame. ».

By default, Blender 3D render frame output resolution is configured at 1920 * 1080 px. Therefore, the texture of our plane can be (for example) the nearest power of 2 square: 2048 * 2048 px.

So, my first challenge was to find the Critical Distance and (why not) to place the plane at that distance in front of the camera.

Critical Distance is obtained by de formula:

 

cos(α2)⋅l2sin(α2){{cos(%alpha over 2)} cdot {d over 2}} over {sin(%alpha over 2)}

Where alpha is the FOV of the camera along X direction and l is the width of the object.

So I coded a simple add-on for Blender (still early alpha: to run it use ALT-P in the Text Editor).

 

# Apparent Dimension To Camera Frame Add-On for Blender 2.70
# Author: Marco Frisan
# Author can be found at: http://endercomics.blogspot.it, http://ilearncocoa.blogspot.it, http://www.facebook.com/marcofrisan

import bpy
import mathutils
import math

def main(context):
for ob in context.scene.objects:
print(ob)

class ApparentDimToCamFrame(bpy.types.Operator):
"""Move or scale and rotate selected object to fit the camera frame with its apparent dimensions."""
bl_idname = "object.apparent_dim_to_cam_frame"
bl_label = "Apparent Dimension To Camera Frame"

@classmethod
def poll(cls, context):
return context.active_object is not None

def execute(self, context):
# Get the active object.
obj = context.active_object

# Get the active object dimensions.
objDim = obj.dimensions
print('Object name: ' + repr(obj.name) + ';nObject dimensions: ' + repr(objDim))

# Get the object transformation matrix in world coordinates.
objWT = obj.matrix_world
print('Object transformation matrix:n' + repr(objWT))

# Get the components of the object transformation.
objLoc, objRot, objSca = objWT.decompose()

# Get the active camera.
# The Scene.camera attribute returns the camera
# as an object of type Object. To get Camera type
# properties you have to get the data block of this
# object. See later.
cam = context.scene.camera
print("Camera type: " + repr(cam.type), ";nCamera name: " + repr(cam.name))

# Get the camera transformation matrix in world coordinates.
camWT = cam.matrix_world
print('Camera transformation matrix:n' + repr(camWT))

# Get the components of the camera transformation.
camLoc, camRot, camSca = camWT.decompose()

# Create a tranlation matrix with the location vector of the camera.
locM = mathutils.Matrix.Translation(camLoc)
print('Camera location matrix:n' + repr(locM))

# Create a rotation matrix with the rotation quaternion of the camera.
rotM = mathutils.Matrix.Rotation(camRot.angle, 4, camRot.axis)
print('Camera rotation matrix:n' + repr(rotM))
#print(rotM)

#print('Camera rotation matrix, pick values:n')
#print('[0][0]:', rotM[0][0])
#print('[1][1]:', rotM[1][1])
#print('[1][2]:', rotM[1][2])
#print('[2][1]:', rotM[2][1])
#print('[2][2]:', rotM[2][2])

# Object must keep its original scale, then we create an identity matrix and set scale values directly.
scaM = mathutils.Matrix.Identity(4)
scaM[0][0] = scaM[1][1] = scaM[2][2] = 1.0
print('Camera scale matrix:n' + repr(scaM))

resultT = locM * rotM * scaM

#print(cam.type)
#print(cam.data.name)
#print(bpy.data.cameras)
#print(bpy.data.cameras.get(cam.data.name))

# Get the camera data block.
camData = bpy.data.cameras.get(cam.data.name)

# Get the camera FOV angle in X direction and divide it by 2.
fovXHalf = camData.angle_x * 0.5

# Get the X dimension of the object.
dimXHalf = objDim[0] * 0.5

# Get the distance at which the object (apparently) fits camera frame.
d = (math.cos(fovXHalf) * dimXHalf) / math.sin(fovXHalf)
print('Critical distance: ' + repr(d))

# Create a vector of magnitude equal to critical distance.
# Since camera always face -Z, the critical distance is the
# third component of the vector.
dV = mathutils.Vector((0.0, 0.0, -d))
print('Critical distance vector: ' + repr(dV))

# Orient the vector like the camera.
dV.rotate(camRot)
print('Critical distance vector aligned to camera: ' + repr(dV))

# Create a translation matrix with the vector.
# NOTE: I think we could use directly a matrix, setting the [3][2] to -d.
# But it works, for now.
dT = mathutils.Matrix.Translation(dV)
print('Critical distance translation matrix:n' + repr(dT))

# Apply the critical distance to the result transformation.
resultT = dT * resultT
print('Result transformation:n' + repr(resultT))

obj.matrix_world = resultT

return {'FINISHED'}

def register():
bpy.utils.register_class(ApparentDimToCamFrame)

def unregister():
bpy.utils.unregister_class(ApparentDimToCamFrame)

if __name__ == "__main__":
register()

# test call
bpy.ops.object.apparent_dim_to_cam_frame()

This is the Blender 3D file: texture_size_&_camera_distance.blend.